IR 017 106 AUTHOR Schlegel, M. J. TITLE Design. · Design Model of Analyze, Design, Development,...
140
DOCUMENT RESUME ED 383 281 IR 017 106 AUTHOR Schlegel, M. J. TITLE A Handbook of Instructional and Training Program Design. PUB DATE [95] NOTE 279p. PUB TYPE Guides Non-Classroom Use (055) Tests/Evaluation Instruments (160) EDRS PRICE MFO1 /PC12 Plus Postage. DESCRIPTORS *Evaluation Methods; *Information Technology; *Instructional Design; Material Development; Needs Assessment; Program Implementation; Summative Evaluation; *Training ABSTRACT Although it is generally argued that information technology (IT) is a science rather than an art, very few professionals follow a process of systematic evaluation and revision, usually due to the level of difficulty and time constraints involved. The intent of this document is to provide a framework for evaluation and revision of instructional design and training via the personal computer, or paper and pencil aids. When implemented, this framework organizes and simplifies the process of evaluation. The generic Design Model of Analyze, Design, Development, Implementation, and Evaluation (ADDIE) is utilized, and detailed job aids are provided in the form of rating sheets and checklists. The five major phases of the design model are discussed: (1) front-end analysis; (2) training needs analysis; (3) instructional design; (4) material development; and (5) implementation. Phase 1 identifies a perceived knowledge deficiency and defines the problem. Phase 2 provides a detailed, relevant list of the skills and knowledge that will make up the content of the training course and a preliminary delivery strategy matching the material to be learned to learner characteristics. Phase 3 outcomes include: determining instructional objectives; developing tests for the objectives; confirming overall delivery strategy; and developing design specifications to used in Phase 4. Phase 4, materials development, considers two major areas, appearance and effect. Feedback from pilot testing is used in the final implementation process, phase 5. Checklists of major steps are provided for each phase. Front-end analysis, training needs analysis, and materials development evaluation checklists are included. The final section, "Summative Evaluation of Training," facilitates the evaluation process through the use of either Kirkpatrick's level of evaluation or the Brethower and Rumler adaptation, and the four levels of training evaluation are outlined. An example training follow-up survey is provided. (Contains 9 references.) (MAS) *********************************************************************** Reproductions supplied by EDRS are the best that can be made from the original document. ***********************************************************************
Transcript of IR 017 106 AUTHOR Schlegel, M. J. TITLE Design. · Design Model of Analyze, Design, Development,...
DOCUMENT RESUME
ED 383 281 IR 017 106
AUTHOR Schlegel, M. J.
TITLE A Handbook of Instructional and Training ProgramDesign.
PUB DATE [95]
NOTE 279p.
PUB TYPE Guides Non-Classroom Use (055) Tests/EvaluationInstruments (160)
EDRS PRICE MFO1 /PC12 Plus Postage.DESCRIPTORS *Evaluation Methods; *Information Technology;
*Instructional Design; Material Development; NeedsAssessment; Program Implementation; SummativeEvaluation; *Training
ABSTRACTAlthough it is generally argued that information
technology (IT) is a science rather than an art, very fewprofessionals follow a process of systematic evaluation and revision,usually due to the level of difficulty and time constraints involved.The intent of this document is to provide a framework for evaluationand revision of instructional design and training via the personalcomputer, or paper and pencil aids. When implemented, this frameworkorganizes and simplifies the process of evaluation. The genericDesign Model of Analyze, Design, Development, Implementation, andEvaluation (ADDIE) is utilized, and detailed job aids are provided inthe form of rating sheets and checklists. The five major phases ofthe design model are discussed: (1) front-end analysis; (2) training
needs analysis; (3) instructional design; (4) material development;
and (5) implementation. Phase 1 identifies a perceived knowledgedeficiency and defines the problem. Phase 2 provides a detailed,relevant list of the skills and knowledge that will make up thecontent of the training course and a preliminary delivery strategymatching the material to be learned to learner characteristics. Phase3 outcomes include: determining instructional objectives; developingtests for the objectives; confirming overall delivery strategy; anddeveloping design specifications to used in Phase 4. Phase 4,materials development, considers two major areas, appearance andeffect. Feedback from pilot testing is used in the finalimplementation process, phase 5. Checklists of major steps areprovided for each phase. Front-end analysis, training needs analysis,and materials development evaluation checklists are included. Thefinal section, "Summative Evaluation of Training," facilitates theevaluation process through the use of either Kirkpatrick's level ofevaluation or the Brethower and Rumler adaptation, and the fourlevels of training evaluation are outlined. An example trainingfollow-up survey is provided. (Contains 9 references.) (MAS)
***********************************************************************
Reproductions supplied by EDRS are the best that can be madefrom the original document.
***********************************************************************
1-"C
) 1
1 0
cc,
U.S
. DE
PA
RT
ME
NT
Of E
DU
CA
TIO
NO
ft(ce
of E
ducs
ioni
o R
esea
rch
and
Impr
ovem
ent
ED
UC
AT
ION
AL
RE
SO
UR
CE
SE
RIC
)INF
OR
MA
TIO
NC
EN
TE
0 T
his
docu
men
t has
bee
n re
prod
uced
as
rece
ived
bor
n th
e pe
rson
or
orga
niza
tion
orig
inat
ing
0 M
inor
Cha
ngeS
hav
e be
en m
od*
10 im
prO
ve
repr
oduc
tion
qual
ity
Poi
nts
of n
ew o
r op
inio
nS S
late
d in
thdO
Ctr
Men
' do
not n
eCes
tanl
y re
priS
tent
ofhcral
OE
RI P
etiti
on o
r po
licy
A H
AN
DB
OO
K O
F IN
STR
UC
TIO
NA
L A
ND
TR
AIN
ING
PRO
GR
AM
DE
SIG
N
A s
impl
e gu
ide
to m
axim
ize
the
desi
gn o
f yo
ur in
stru
ctio
n gi
ven
your
tim
e an
d/or
bud
get c
onst
rain
ts.
A "
HO
W T
O"
appl
icat
ion
of th
e la
test
in I
nstr
uctio
nal D
esig
n Pr
inci
ples
to e
nhan
ce th
e ut
ility
(i.e
. the
effi
cien
cy a
nd e
ffec
tiven
ess)
of
your
pro
gram
s th
roug
h th
e ap
plic
atio
nof
qua
lity
cont
ra",
to a
chie
ve
cont
inuo
us im
prov
emen
t.
by M.J
.Sch
lege
lM
.Ed.
MB
A
2B
ES
T C
OP
Y A
VA
ILA
BLE
PE
RM
ISS
ION
TO
RE
PR
OD
UC
E T
HIS
MA
TE
RIA
L H
AS
BE
EN
GR
AN
TE
D B
Y
Michael J. Schlegel
TO
TH
E E
DU
CA
TIO
NA
L R
ES
OU
RC
ES
INF
OR
MA
TIO
N C
EN
TE
R (
ER
IC)
Tab
le O
f C
onte
nts
Intr
oduc
tion
Inst
ruct
iona
l Tec
hnol
ogy
as a
Sci
ence
1
Fron
t End
Ana
lysi
s2
Tra
inin
g N
eeds
Ana
lysi
s3
Inst
ruct
iona
l Des
ign
3
Mat
eria
l Dev
elop
men
t4
Impl
emen
tatio
n5
Sum
mat
ive
Eva
luat
ion
6
Sum
mar
y6
Wor
kshe
ets
Des
crip
tion
Fron
t End
Ana
lysi
s8
Dat
a fo
r T
rain
ing
Req
uest
s13
Dat
a C
olle
ctio
n in
Fro
nt E
nd A
naly
sis
14
Wha
t to
do if
Tra
inin
g is
not
Con
firm
ed16
Fron
t End
Ana
lysi
s E
valu
atio
n17
Fron
t End
Ana
lysi
s R
epor
t19
Tra
inin
g N
eeds
Ana
lysi
sD
escr
iptio
n T
rain
ing
Nee
ds A
naly
sis
21
Prel
imin
ary
Del
iver
y St
rate
gy22
Che
cklis
t of
Act
iviti
es D
urin
g T
rain
ing
Nee
ds A
naly
sis
24D
ata
Col
lect
ion
in T
ask
Ana
lysi
s25
Dat
a C
olle
ctio
n in
Lea
rner
Ana
lysi
s28
Tra
inin
g N
eeds
Ana
lysi
s E
valu
atio
n30
Tas
k D
etai
ls32
How
to D
eriv
e Sk
ill a
nd K
now
ledg
e R
equi
rem
ents
35
45
Tab
le O
f C
onte
nts
Inst
ruct
iona
l Des
ign
Des
crip
tion
Inst
ruct
iona
l Des
ign
39C
heck
list o
f A
ctiv
ities
Dur
ing
Inst
ruct
iona
l Des
ign
41
Dev
elop
ing
Inst
ruct
iona
l Obj
ectiv
es42
Edi
ting
Inst
ruct
iona
l Obj
ectiv
es43
Obj
ectiv
e T
ests
44H
ow to
Sel
ect t
he O
ptim
um D
eliv
ery
Stra
tegy
48T
rain
ing
Del
iver
y St
rate
gies
51
Med
ia S
elec
tion
54
Sele
ctin
g In
stru
ctio
nal M
etho
ds55
Des
ign
Spec
ific
atio
ns R
epor
t60
Eve
nts
of I
nstr
uctio
n61
Lan
guag
e R
evie
w62
Tes
t Spe
cifi
catio
ns63
Mat
eria
l Dev
elop
men
tD
escr
iptio
n M
ater
ial D
evel
opm
ent
65
Che
cklis
t of
Act
iviti
es D
urin
g M
ater
ial D
evel
opm
ent
68In
stru
ctio
nal M
ater
ial R
evie
w69
How
to C
ondu
ct a
Wal
k-T
hrou
gh74
.
How
to C
ondu
ct a
Dev
elop
men
tal T
est
75R
evis
ions
Bas
ed o
n D
evel
opm
enta
l Tes
t Fee
dbac
k79
Mat
eria
ls D
evel
opm
ent E
valu
atio
n82
Impl
emen
tatio
nD
escr
iptio
n Im
plem
enta
tion
88
Che
cklis
t of
Act
iviti
es D
urin
g Im
plem
enta
tion
90In
stru
ctio
nal T
echn
olog
y R
evie
w91
How
to I
ntro
duce
a C
ours
e94
ii
Tab
le O
f C
onte
nts
How
to P
repa
re I
i.,.r
ucto
rs96
How
to P
repa
re I
nstr
uctio
nal M
ater
iais
97H
ow to
Pre
pare
Fac
ilitie
s an
d E
quip
men
t99
Dat
a C
olle
ctio
n in
Fol
low
-Up
Eva
luat
ion
100
Cou
rse
Mai
nten
ance
Che
cklis
t10
3
Eva
luat
ion
of T
rain
ing
Intr
oduc
tion
Sum
mat
ive
Eva
luat
ion
108
Plan
ning
110
Philo
soph
y of
Eva
luat
ion
111
Lev
els
of E
valu
atio
n11
1
Lev
el T
wo
112
Lev
el T
hree
.....
....
....
......
113
Lev
el F
our
114
Sum
mar
y11
5
Intr
oduc
tion
Form
ativ
e E
valu
atio
n11
5
Tra
inin
g Fo
llow
-Up
Surv
ey11
6
8III
Intr
oduc
tion
It is
arg
ued
that
Ins
truc
tiona
l Tec
hnol
ogy
(IT
) is
a s
cien
ce r
athe
r th
an a
n ar
t lar
gely
bec
ause
of
the
use
of a
n In
stru
ctio
nal
Mod
el a
nd th
e fe
atur
es o
f ev
alua
tion
and
revi
sion
of
inst
ruct
ion
inco
rpor
ated
in th
e m
odel
s. A
lthou
gh m
ost p
eopl
e ag
ree
with
the
abov
e ve
ry f
ew, i
f an
y fo
llow
a p
roce
ss o
f sy
stem
atic
eva
luat
ion
and
revi
sion
for
a v
arie
ty o
f re
ason
s. M
inim
ally
, mos
t of
us (
this
auth
or in
clud
ed)
prov
ide
for
a su
mm
ativ
e ev
alua
tion
of r
esul
ts i.
e. L
evel
One
eva
luat
ions
and
som
e so
rt o
f Fo
rmat
ive
Eva
luat
ion
usua
lly a
"cr
uise
" of
you
r m
ater
ial b
y a
peer
, bos
s or
eve
n sp
ouse
. Per
haps
the
two
pred
omin
ant r
easo
n's
for
the
negl
igen
ce a
re
the
leve
l of
diff
icul
ty a
nd ti
me
cons
trai
nt in
volv
ed. I
roni
cally
, the
ver
y re
ason
that
mak
e IT
a s
cien
ce a
nd n
ot a
n ar
t are
oft
en
viol
ated
and
the
scie
nce
vers
es a
rt a
rgum
ent t
hus
cont
inue
s in
to p
erpe
tuity
. With
the
adve
nt o
f te
chno
logy
and
in p
artic
ular
the
pers
onal
com
pute
r th
is n
eglig
ence
for
the
two
"big
" re
ason
s is
and
sho
uld
be u
nacc
epta
ble
(Cri
min
al N
eglig
ence
?).
The
inte
nt o
f th
is d
ocum
ent i
s to
pro
vide
a f
ram
ewor
k fo
r ev
alua
tion
and
revi
sion
via
the
pers
onal
com
pute
r or
pap
er a
nd
penc
il jo
b ai
d's
that
whe
n im
plem
ente
d sh
ould
org
aniz
e an
d si
mpl
ify
the
proc
ess
and
even
tual
ly te
rmin
ate
the
argu
men
t, an
d ha
ve
IT a
ccep
ted
pred
omin
atel
y as
the
scie
nce
that
it is
.
Thi
s pa
per
will
util
ize
the
gene
ric
Inst
ruct
iona
l Des
ign
Mod
el o
f A
naly
ze, D
esig
n, D
evel
opm
ent,
Impl
emen
tatio
n an
d
Eva
luat
ion
(AD
DIS
see
nex
t pag
e) a
nd p
rovi
de d
etai
led
job
aids
in th
e fo
rm o
f ra
ting
shee
ts a
nd c
heck
lists
(m
echa
nica
lly o
r ha
rd
copy
) fo
r ea
ch o
f th
e m
ajor
ste
ps.
11
10
.
rri1
77.;
5:=
.,5'
-,
5 -.
...,..
..55;
54,
5555
555:
.M5:
::',',
5':"
.555
550:
::45:
::::::
" 55
, <55
.5":
:=5'
5555
:,.::.
's,'K
.:5,,,
,<,"
/
Inst
ruct
iona
l Des
ign
Mod
el
5";5
5`;
.,:'..
4./,.
.-..5
.(.,
, ,:.5
,, ,
,V
t.". '
Kft
z..-
-;...
.:;.'5
`:,.
.,..F
,R.r
flp
r' if
f
"5"
',4
F?, '
e' /
...:*
1,4;
`,
''"4"
,?,1
"4
,, ,,,
,,,,,,
it
;2,"
";,5
-1,'.
5.7,
'",
::,,
5`,"
,5
4
12
5:55
BE
ST
CO
PY
AV
AIL
AB
LE13
Thr
ough
our
ow
n ap
plic
atio
n of
tech
nolo
gy th
e ve
ry te
chno
logy
we
prof
ess
to o
ur tr
aini
ng c
lient
ele
and
som
e pr
actic
e
with
thes
e jo
b ai
ds, w
e ca
n pr
oper
ly f
ollo
w th
e m
odel
and
in a
tim
ely
man
ner,
whi
ch is
con
sist
ent w
ith r
eal w
orld
con
stra
ints
The
follo
win
g is
a b
rief
dis
cuss
ion
of th
e cr
itica
l ste
ps in
the
five
maj
or p
hase
s.
Fron
t-E
nd A
naly
sis
Thi
s ph
ase
inco
rpor
ates
a F
ront
-End
Ana
lysi
s as
wel
l as
a Pe
rfor
man
ce A
naly
sis
in w
hich
pot
entia
l or
actu
al p
erfo
rman
ce
defi
cien
cies
whi
ch a
re s
uspe
cted
nee
d to
be
anal
yzed
. I w
ill n
ot c
over
Fro
nt-E
nd a
naly
sis
in a
ny d
etai
l but
the
inte
rest
ed r
eade
r ca
n
cons
ult T
hom
as G
ilber
ts (
1979
) "B
ehav
iora
l Eng
inee
ring
Mod
el",
or
Rum
mle
r an
d B
rach
e's
(198
1) m
odel
.
Of
part
icul
ar n
ote
this
pha
se s
tart
s w
ith a
req
uest
of
som
e so
rt to
inve
stig
ate
a pe
rcei
ved
defi
cien
cy it
doe
s no
t usu
ally
antic
ipat
e fu
ture
pro
blem
s (n
ew e
mpl
oyee
s )
or s
kills
(ne
w T
echn
olog
ies)
that
mig
ht b
e ne
eded
i.e.
Pro
activ
e T
rain
ing
or
Edu
catio
n an
d D
evel
opm
ent w
hich
is e
valu
ated
dif
fere
ntly
and
not
pan
. of
this
pap
er. T
he p
roce
ss to
be
follo
wed
in th
is s
tep
is
defi
ned
in th
e fi
rst s
ectio
n "F
ront
-End
Ana
lysi
s E
valu
atio
n"
The
pri
mar
y ou
tput
's a
re a
Pro
blem
Ana
lysi
s w
ith a
cle
arly
def
ined
pro
blem
. Doe
s a
prob
lem
exi
st a
nd is
it w
orth
sol
ving
by
trai
ning
or
othe
rwis
e? o
r no
trai
ning
or
solu
tion
is w
orth
the
cost
. Thi
s in
turn
will
dep
end
on th
e an
alys
es a
nd d
isco
very
of
wha
t
kind
of
prob
lem
it is
:
1.T
hey
do n
ot k
now
how
to p
erfo
rm c
orre
ctly
2T
hey
don'
t wan
t to
perf
orm
cor
rect
ly (
mot
ivat
ion)
15
142
3.So
met
hing
is p
reve
ntin
g th
em f
rom
per
form
ing
corr
ectly
Onl
y in
the
case
of
#1 a
bove
is th
ere
clea
r ju
stif
icat
ion
to c
ontin
ue w
ith th
e T
rain
ing
and
Dev
elop
men
t Pro
cess
. The
oth
er
outp
ut o
f th
e Fr
ont-
End
Ana
lysi
s is
a R
ecom
men
ded
solu
tion;
Tra
inin
g or
Non
- T
rain
ing.
The
nov
ice
prac
titio
ner
shou
ld b
e w
ary
of v
entu
ring
into
oth
er a
reas
whe
re h
e ha
s lit
tle o
r no
exp
ertis
e he
re. I
t may
not
be
polit
ical
ly a
stut
e to
cla
im a
noth
er d
epar
tmen
t
is d
efic
ient
. The
se O
utpu
ts th
en b
ecom
e th
e in
puts
of
the
next
ste
p. A
t thi
s po
int c
are
mus
t be
take
n no
t to
mak
e on
e of
the
com
mon
mis
take
s B
reth
ower
and
Rum
ler
(197
6.)
:1.
To
take
the
inpu
t of
line
man
agem
ent w
ithou
t que
stio
n an
d ag
ree
to
deve
lop
trai
ning
with
out a
ny r
eal e
vide
nce
that
it is
req
uire
d. 2
. To
agre
e on
for
m a
nd le
ngth
of
a tr
aini
ng c
ours
e lo
ng b
efor
e
read
y. T
he r
esul
t is
ofte
n a
cour
se th
at d
oes
not m
eet t
he n
eeds
and
is to
exp
ensi
ve a
nd in
effi
cien
t.
Tra
inin
g N
eeds
Ana
lysi
s
The
goa
ls o
f th
is p
hase
are
to p
rodu
ce a
det
aile
d, r
elev
ant l
ist o
f th
e sk
ills
and
know
ledg
e th
at w
ill m
ake
up th
e co
nten
t of
the
cour
se a
nd a
pre
limin
ary
deliv
ery
stra
tegy
mat
chin
g th
e m
ater
ial t
o be
lear
ned
to th
e le
arne
r ch
arac
teri
stic
s. L
earn
er a
naly
sis
(see
"T
rain
ing
Nee
ds A
naly
sis
Eva
luat
ion"
pg.
TN
A 1
0-11
.) is
fir
st c
ondu
cted
to d
eter
min
e th
e tr
aini
ng p
opul
atio
ns c
hara
cter
istic
s
and
entr
y le
vel s
kills
Thi
s in
put i
s th
en u
sed
to d
eter
min
ed a
pre
limin
ary
deliv
ery
stra
tegy
("T
rain
ing
Del
iver
y St
rate
gies
"pg.
17-
19. )
Inst
ruct
iona
l Des
ign
The
ana
lysi
s w
e ju
st c
ompl
eted
can
be
used
as
a bl
uepr
int t
o cr
eate
Tra
inin
g. A
lthou
gh it
is te
mpt
ing
to s
tart
dev
elop
ing
mat
eria
ls y
ou p
ay f
or it
in th
e lo
ng r
un b
y in
clud
ing
irre
leva
nt c
onte
nt a
nd e
xces
sive
rev
isio
ns th
at p
rope
r in
stru
ctio
nal d
esig
n
wor
k w
ould
hav
e el
imin
ated
.
171.
63
The
maj
or o
utco
mes
of
this
piv
otal
pha
se in
clud
e : d
eter
min
ing
inst
ruct
iona
l obj
ectiv
es, d
evel
opin
g te
sts
for
the
obje
ctiv
es, (
see
"Edi
ting
Inst
ruct
iona
l Obj
ectiv
es"p
g. D
esig
n 6
and
"Med
ia S
elec
tion"
pg.
Des
ign
14)
conf
irm
ing
over
all d
eliv
ery
stra
tegy
incl
udin
g m
edia
sel
ectio
n an
d de
velo
ping
des
ign
spec
ific
atio
ns to
be
used
in th
e ne
xt p
hase
mat
eria
l dev
elop
men
t. B
efor
e
proc
eedi
ng to
mat
eria
l dev
elop
men
t the
des
igne
r m
ust b
e sa
tisfi
ed th
at: t
he m
edia
sel
ecte
d ar
e ap
prop
riat
e fo
r th
e ob
ject
ives
, the
lear
ner,
the
desi
gn a
nd th
e de
liver
y fa
ctor
s. T
hat t
he le
arni
ng a
ctiv
ities
are
bas
ed o
n th
e ev
ents
of
inst
ruct
ion
(see
pg.
Des
ign
21
"Eve
nts
Of
Inst
ruct
ion"
)and
are
con
sist
ent w
ith th
e ob
ject
ives
.
The
mos
t cri
tical
ste
p of
the
entir
e pr
oces
s is
in th
e de
term
inat
ion
of in
stru
ctio
nal o
bjec
tives
and
inte
ract
ion
with
the
clie
nt
is im
pera
tive
in o
rder
to a
ssur
e th
at: 1
) T
he o
bjec
tives
are
app
ropr
iate
to m
eet t
he c
lient
s ne
ed. 2
) T
he p
rogr
am is
des
igne
d to
mee
t the
obj
ectiv
e. T
rain
ing
that
is n
ot f
ully
col
labo
rativ
e is
like
ly to
be
ster
ile, a
nd in
effe
ctiv
e.
Mat
eria
l Dev
elop
men
t
App
eara
nce
(see
pg.
MD
18-
.19
"Mat
eria
l Dev
elop
men
t Eva
luat
ion"
) an
d ef
fect
of
all i
nstr
uctio
nal m
ater
ials
is u
ltim
atel
y
your
res
pons
ibili
ty. T
his
incl
udes
the
stud
ent a
ndin
stru
ctio
nal g
uide
. As
an a
id to
rev
iew
the
fina
l pro
duct
and
bef
ore
impl
emen
tatio
n th
e "I
nstr
uctio
nal T
echn
olog
y R
evie
w",
(se
e pg
. IM
P 3)
shou
ld b
e co
mpl
eted
.
The
fin
ishe
d pr
oduc
t now
pro
duce
d ne
eds
to b
e ch
ecke
d fo
r fl
ow th
at is
a y
es a
nsw
er m
ust b
e gi
ven
to th
e si
x "f
inal
"
ques
tions
on
the
"Con
grue
ncy
Che
ck",
(se
e pg
. IM
P 4-
5) o
r a
criti
cal l
ink
may
be
mis
sing
.
1 9
184
Impl
emen
tatio
n
Plan
ning
for
the
pilo
t tes
ting
of th
e co
urse
sho
uld
occu
r as
ear
ly a
s po
ssib
le in
the
trai
ning
dev
elop
men
t pro
cess
.
Res
pons
ibili
ty f
or a
ll ne
cess
ary
envi
ronm
enta
l ite
ms
such
as
room
and
equ
ipm
ent n
eed
to b
e de
term
ined
and
doc
umen
ted.
Sche
dulin
g of
the
initi
al o
ffer
ing
and
freq
uenc
y an
d du
ratio
n of
cla
sses
onc
e th
e co
urse
is b
roug
ht o
n-st
ream
als
o ne
ed to
be
agre
ed to
. You
nee
d to
wor
k cl
osel
y w
ith th
e cl
ient
,SM
E's
and
oth
ers
that
nee
d to
be
pres
ent i
n or
der
to a
ssur
e th
e co
urse
can
be d
eliv
ered
imm
edia
tely
upo
n co
mpl
etio
n. T
he p
redo
min
ant p
art o
f th
e au
dien
ce s
houl
d be
mem
bers
of
the
targ
et a
udie
nce.
The
desi
gner
nee
ds to
be
pres
ent f
or th
e in
itial
off
erin
g in
ord
er to
gat
her
real
tim
e fe
edba
ck f
rom
the
part
icip
ants
and
doc
umen
t any
chan
ges
that
may
be
need
ed. D
urin
g im
plem
enta
tion
clue
s to
look
for
incl
ude
Abe
lla (
1986
) :
1. P
artic
ipan
ts ta
lkin
g or
not
talk
ing
abou
t the
pro
gram
. 2. N
on-p
artic
ipan
ts n
ot b
eing
enc
oura
ged
to jo
in in
con
vers
atio
ns. 3
. Man
y cy
nica
l rem
arks
. 4. P
artic
ipan
ts
not d
oing
ass
ignm
ents
. 5. P
artic
ipan
ts a
re la
te a
t sta
rt o
r br
eaks
or
skip
out
alto
geth
er. 6
. Par
ticip
ants
do
not a
sk f
or h
elp
or
addi
tiona
l inf
orm
atio
n or
idea
s. 7
. Hum
orou
s re
mar
ks m
ade
in c
lass
that
hav
e no
con
nect
ion
to c
lass
act
iviti
es. A
ll th
ese
may
be
sign
s of
impe
ndin
g pr
oble
ms
that
nee
d fu
rthe
r in
vest
igat
ion.
Adj
ustm
ents
hav
ing
been
mad
e it
is n
ow ti
me
for
the
full
rele
ase
of
the
prog
ram
. Thi
s sh
ould
onl
y be
don
e w
hen
the
follo
win
g ha
ve b
een
answ
ered
to th
e cl
ient
s an
d yo
ur s
atis
fact
ion:
1. W
as th
e or
igin
al n
eed
accu
rate
ly d
efin
ed?
2. W
ere
the
obje
ctiv
es a
ppro
pria
te to
mee
t the
nee
d?
3. D
oes
the
prog
ram
des
ign
achi
eve
the
obje
ctiv
e?
4. I
s th
e co
nten
t app
ropr
iate
to th
e ob
ject
ive?
5. A
re th
e m
ater
ials
of
good
qua
lity?
inte
rest
ing
and
wel
l wri
tten?
2021
5
6 D
o th
e m
ater
ials
do
wha
t the
des
ign
inte
nds
them
to d
o?
Sum
mat
ive
Eva
luat
ion
Of
Tra
inin
g
The
mos
t use
ful m
odel
of
eval
uatio
n I
have
fou
nd is
Kir
kpat
rick
s' (
1975
) le
vel o
f ev
alua
tion
or th
e ad
apta
tion
by
Bre
thow
er a
nd R
umle
r (1
976)
. The
tool
can
be
used
as
follo
ws:
Lev
el 1
Are
the
trai
nees
hap
py?
Lev
el 2
Doe
s th
e tr
aini
ng d
o
wha
t it i
s su
ppos
ed to
do?
Lev
el 3
Are
the
conc
epts
use
d (
see
pg. S
E 8
-10.
) L
evel
4 D
oes
appl
icat
ion
of th
e co
ncep
ts a
ffec
tth
e
orga
niza
tion?
For
eac
h of
thes
e le
vels
the
who
- s
ourc
e of
dat
a, w
hat -
to m
easu
re, w
here
- f
ind
data
, why
- c
rite
ria,
and
how
-
data
gat
heri
ng m
etho
d m
ust b
e an
swer
ed.
Pepp
er (
1984
) th
e ro
le o
f th
e tr
aini
ng f
unct
ion
with
res
pect
to f
ollo
w u
p is
to a
sses
s th
e ex
tent
the
new
idea
s, s
kills
and
mot
ivat
ion
will
get
a n
atur
al o
ppor
tuni
ty f
or e
xerc
ise
and
rein
forc
emen
t. A
nd h
avin
g do
ne th
is to
rec
omm
end
step
s to
ove
rcom
e
any
obst
acle
s T
his
tend
s to
be
over
look
ed b
y tr
aini
ng w
hich
oft
en o
mits
to c
ondu
ct a
ny f
ollo
w u
p. W
e te
nd to
look
at t
he
com
plet
ion
of a
pro
gram
as
the
end
of r
espo
nsib
ility
. Thi
s at
titud
e in
dica
tes
insu
ffic
ient
und
erst
andi
ng o
f th
e le
arni
ng p
roce
ss.
One
has
to b
e th
ankf
ul th
at th
e na
tura
l inc
linat
ions
of
peop
le to
see
k re
info
rcem
ent o
n th
eir
own
with
out w
aitin
g fo
r in
tere
st to
be
show
n by
oth
ers
.
Sum
mar
y
If d
one
wel
l, de
sign
ing
and
deve
lopi
ng tr
aini
ng r
equi
re a
gre
at d
eal o
f sk
ill a
nd k
now
ledg
e. T
he d
esig
ner
need
s to
gat
her
expe
rts
in in
stru
ctio
nal d
evel
opm
ent,
subj
ect m
atte
r, p
rodu
ctio
n, c
ompu
ter
and
med
ia s
kills
. Mor
eove
r, a
ll th
is m
ust b
e co
mpl
eted
on ti
me
and
with
in b
udge
t. H
e/Sh
e m
ust a
lway
s as
sure
to th
e be
st o
f th
eir
abili
ty th
at th
e pr
ojec
t add
ress
es th
e pe
rfor
man
ce
2223
6
prob
lem
, mee
ts th
e tr
ue n
eeds
of
the
targ
et p
opul
atio
n, s
atis
fies
you
r cl
ient
, and
is c
onsi
sten
t with
the
goal
s of
the
Com
pany
.
The
se jo
b ai
ds (
mec
hani
cal o
r ot
her-
wis
e) a
re d
esig
ned
to g
reat
ly s
impl
ify
the
proc
ess.
25
247
PER
FOR
MA
NC
E/F
RO
NT
-EN
D A
NA
LY
SIS
From
a m
anag
emen
t per
spec
tive
it m
ight
app
ear
that
we
are
addi
ng a
lot o
f ex
tra
time
to th
e pr
ojec
t. U
pon
furt
her
exam
inat
ion,
the
valu
e of
Per
form
ance
/Fro
nt-E
nd A
naly
sis
is th
at w
hen
you
com
plet
e Pe
rfor
man
ce/F
ront
-End
Ana
lysi
s an
d de
cide
to g
o on
to n
ext p
hase
wha
teve
r da
ta y
ou c
olle
ct is
stil
l rel
evan
t whe
n yo
u ge
t to
trai
ning
nee
ds a
naly
sis.
Als
o it
is o
rdin
arily
com
plet
ed in
a s
hort
per
iod
of ti
me.
The
inte
nt is
to m
ake
a de
term
inat
ion
as q
uick
ly a
s po
ssib
le a
nd m
ove
on .
Exp
erie
nce
has
show
n th
at th
e al
tern
ativ
e, n
ot d
oing
Per
form
ance
/Fro
nt-E
nd A
naly
sis,
you
may
be
wel
l int
o de
taile
d an
alys
is b
efor
e yo
u re
aliz
e
trai
ning
is n
ot th
e on
ly o
r be
st w
ay, t
hus
was
ting
muc
h m
ore
time.
The
con
cept
is u
sed
in o
ther
are
as o
f bu
sine
ss a
s w
ell,
they
will
tell
you
that
bef
ore
they
com
mit
dolla
rs o
r re
sour
ces
they
do
the
sam
e ty
pe o
f an
alys
is. T
his
prod
uces
a h
ighe
r qu
ality
pro
duct
at
low
er c
osts
. Fur
ther
mor
e Pe
rfor
man
ce/F
ront
-End
Ana
lysi
s pr
ovid
es th
e ba
sis
by w
hich
trai
ning
is e
valu
ated
alth
ough
it is
not
the
only
one
. One
goo
d m
easu
re o
f ef
fect
iven
ess
is th
e de
gree
to w
hich
the
prob
lem
s id
entif
ied
in P
erfo
rman
ce/F
ront
-End
Ana
lysi
s
have
impr
oved
the
proc
ess.
With
out t
he id
entif
icat
ion
of th
ose
prob
lem
s an
d/or
cos
t red
uctio
ns w
e w
ould
hav
e a
grea
t dea
l of
diff
icul
ty e
valu
atin
g th
e tr
aini
ng's
eff
ectiv
enes
s. T
he b
otto
m li
ne is
that
in th
e lo
ng r
un th
e co
mpa
ny w
ill s
ave
sign
ific
antly
by
com
plet
ing
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
as th
e fi
rst s
tep
of e
ach
trai
ning
pro
ject
.
2327
8
The
Cou
rse
Man
ager
/Tea
cher
initi
ates
the
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
and
mus
t ins
ist t
hat t
rain
ing
proj
ects
will
be
unde
rtak
en o
nly
whe
n a
clea
r ne
ed f
or tr
aini
ng h
as b
een
iden
tifie
d, a
nd w
here
the
bene
fits
of
trai
ning
just
ify
the
cost
s. I
n ad
ditio
n,
whe
n ot
her
mea
ns h
ave
been
iden
tifie
d th
ey w
ill b
e co
nsid
ered
soo
ner
and
with
bet
ter
data
for
whi
ch to
mak
e a
reco
mm
enda
tion.
In th
is p
hase
it is
the
ultim
ate
resp
onsi
bilit
y of
the
Cou
rse
Man
ager
/Tea
cher
to d
eter
min
e:
IIf
ther
e is
just
ific
atio
n to
con
tinue
with
trai
ning
dev
elop
men
t?
2. I
f so
me
othe
r so
lutio
n is
app
ropr
iate
and
info
rmin
g an
d pr
ovid
ing
all p
ertin
ent i
nfor
mat
ion.
3. N
o so
lutio
n is
wor
th th
e ef
fort
.
The
rol
e of
the
Cou
rse
Man
ager
/Tea
cher
is to
mak
e su
re th
e an
alys
is is
com
plet
ed o
n tim
e, a
nd in
a p
rope
r m
anne
r, a
nd to
serv
e as
dev
ils a
dvoc
ate
whe
n re
view
ing
the
find
ings
. The
Cou
rse
Man
ager
/Tea
cher
ass
ures
that
con
clus
ions
are
just
ifie
d by
the
data
aft
er a
ll it
is h
e w
ho a
rran
ges
for
com
pany
com
mitm
ent t
o th
e pr
ojec
t.
Bef
ore
proc
eedi
ng to
the
next
pha
se th
e C
ours
e M
anag
er/T
each
er s
houl
d pr
epar
e w
ith th
e ap
prop
riat
e st
aff
mem
ber
a
"Per
form
ance
/Fro
nt-E
nd A
naly
sis
Rep
ort"
whi
ch in
clud
es th
e fo
llow
ing:
1 W
ho th
e ta
rget
per
form
ers
are?
and
thei
r lo
catio
n an
d jo
b tit
les.
2 H
ow m
any
peop
le w
ere
invo
lved
and
for
how
long
?
3. W
hat p
robl
em in
dica
tors
sug
gest
the
need
for
the
proj
ect?
2928
9
4. W
hat m
etho
ds in
stru
men
ts w
ere
used
to c
olle
ct th
e da
ta?
5. W
hat i
s th
e id
eal p
erfo
rman
ce v
s. a
ctua
l and
wha
t is
the
caus
e?
6. W
hat o
ther
sol
utio
ns w
ere
cons
ider
ed a
nd w
hy?
7. W
hat i
s th
e ra
tiona
le f
or y
our
choi
ce ?
The
fin
al s
tep
is th
e pr
epar
atio
n an
d re
view
of
the
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis r
epor
t. A
per
form
ance
rep
ort o
n th
e
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis.
As
follo
ws:
I. P
lann
ing
A. C
ompr
ehen
sive
1. D
oes
the
Plan
for
Per
form
ance
/Fro
nt-E
nd A
naly
sis
spec
ify:
a. r
atio
nale
b. t
arge
t gro
up c
. dat
a re
quir
emen
ts
2. S
tep
by s
tep
activ
ities
B R
ealis
tic I) D
oes
the
plan
take
into
con
side
ratio
n th
e na
ture
and
sco
pe
of th
e ap
pare
nt p
robl
em?
2) C
onsi
dere
d C
onst
rain
ts
a. o
rgan
izat
iona
l b. p
erso
nnel
c. t
ime
d. r
esou
rce
313
010
II. C
lear
ly d
efin
ed P
robl
em
A. R
elia
ble
and
valid
dat
a
B. A
ctua
l vs.
Des
ired
per
form
ance
C. O
ther
cau
ses
of p
robl
em
III.
Sol
utio
ns a
nd r
ecom
men
datio
ns
A. D
ata
revi
ewed
by
othe
rs w
ho u
nder
stan
d th
e pr
oble
m
1) C
lient
2) S
ubje
ct M
atte
r E
xper
t
B. H
ave
reas
onab
le s
olut
ions
bee
n ch
ecke
d
C. I
s fi
nal s
olut
ion
best
pos
sibl
e al
tern
ativ
e
IV. F
easi
bilit
y
A. D
oes
the
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
cont
ain
adeq
uate
info
rmat
ion
to m
ake
a de
cisi
on f
or p
roje
ct c
ontin
uatio
n or
othe
r so
lutio
n?
B. D
o su
ffic
ient
res
ourc
es e
xist
to w
ork
on th
e so
lutio
n to
the
prob
lem
?
3233
11
V. C
osts
and
Ris
ks
A. A
re th
e co
sts
and
risk
s of
the
solu
tions
iden
tifie
d an
d ac
cura
te?
B. H
ave
the
risk
s an
d co
st o
f ig
nori
ng th
e pr
oble
m b
een
cons
ider
ed?
At t
his
poin
t the
Cou
rse
Man
ager
/Tea
cher
mus
t tak
e ca
re n
ot to
mak
e on
e of
the
com
mon
mis
take
s:
1. T
o ta
ke th
e in
put o
f lin
e
man
agem
ent w
ithou
t que
stio
n an
d ag
ree
to d
evel
op tr
aini
ng w
ithou
t any
rea
l evi
denc
e th
at it
is r
equi
red.
2. T
o ag
ree
on f
orm
and
leng
th o
f a
trai
ning
cou
rse
long
bef
ore
read
y. T
he r
esul
t is
ofte
n a
cour
se th
at d
oes
not m
eet t
he n
eeds
and
is to
exp
ensi
ve a
nd
inef
ficie
nt.
3431
;
Perf
orm
ance
Ana
lysi
s E
valu
atio
n
Plan
ning
Com
preh
ensi
veR
ealis
tic
Cle
ar d
efin
ition
of
the
prob
lem
Rel
evan
t def
icits
4V
.1, :
4...1
4191
.01/
;Sre
M3:
0 'W
e ''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''''''''
''.k
ff
gfil
Wp4
; ,"4
eW
44,
--; C
ompr
ehen
sive
sol
utio
ns
36
.:;;;;
*::::
;..:::
:.
'
37
IF y
our
clie
ntgi
ves
this
rea
son
for
requ
estin
gtr
aini
ng
Thi
s jo
b ne
eds
trai
ning
(fo
rex
istin
g, m
odif
ied,
or n
ew jo
bs)
Thi
s is
a n
ewsy
stem
(or
met
hod
oreq
uipm
ent)
Exi
stin
g tr
aini
ng is
inad
equa
te
Dat
a fo
r T
rain
ing
Req
uest
s
TH
EN
you
hav
e to
col
lect
dat
a to
ans
wer
thes
e ty
pes
ofqu
estio
ns.
Wha
t are
the
job
rela
ted
prob
lem
s in
term
s of
out
puts
, eff
ects
on
othe
r w
ork
grou
ps, s
ervi
ce to
the
cust
omer
s, lo
ss o
f re
venu
e to
the
regi
on o
r to
the
com
pany
, etc
.?
Wha
t are
the
diff
eren
ces
betw
een
the
old
and
the
new
?
Wha
t dif
ficu
lties
do
we
antic
ipat
e? W
hy?
Wha
t are
the
job
rela
ted
prob
lem
s in
term
s of
out
puts
, eff
ects
on
othe
r w
ork
grou
ps, s
ervi
ce to
the
cust
omer
s, lo
ss o
f re
venu
e to
the
regi
on o
r to
the
Com
pany
, etc
.?
Wha
t par
ts o
f th
e co
urse
are
obs
olet
e?W
hat p
arts
of
the
cour
se a
re in
accu
rate
?W
hat p
arts
of
the
cour
se a
re m
issi
ng?
We
have
a n
ew ty
pe o
f st
uden
tW
hat a
re th
e ac
tual
dif
ficu
lties
in tr
aini
ngW
hat a
re th
e ac
tual
pro
blem
s in
term
s of
out
puts
?
3839
13
Dat
a C
olle
ctio
n in
Per
form
ance
/Fro
nt-E
nd A
naly
sis
To
answ
er th
is q
uest
ion
Use
thes
e so
urce
sA
nd th
ese
stra
tegi
es
Wha
t is
the
appa
rent
pro
blem
?C
lient
Wha
t are
the
char
acte
rist
ics
of th
e ta
rget
gro
up?
Perf
orm
er, S
uper
viso
r, T
rain
er, C
lient
Job
Des
crip
tion
Whe
n is
the
solu
tion
need
ed?
Wha
t tra
inin
g is
cur
rent
ly a
vaila
ble?
Doe
s th
e pe
rfor
man
ce p
robl
em r
eally
exi
st?
Wha
t are
the
idea
ls a
nd a
ctua
ls(a
ntic
ipat
ed o
r re
al)?
Wha
t are
the
maj
or c
ause
s of
the
prob
lem
?
0
Clie
nt
Tra
iner
, Tra
inin
g O
rgan
izat
ion
Perf
orm
er, S
uper
viso
r, R
elat
ed W
ork
Gro
ups,
Clie
nt, C
usto
mer
Out
put
Prod
uctiv
ity R
ecor
ds, O
ther
Raw
Dat
a (e
.g.,
loss
of
reve
nue)
Perf
orm
er, S
uper
viso
r, R
elat
ed W
ork
Gro
ups,
Clie
nt, O
ther
Exp
erts
Out
put/P
rodu
ctiv
ity R
ecor
ds
Perf
orm
er, S
uper
viso
r, R
elat
ed W
ork
Gro
ups,
Clie
nt
Inte
rvie
w
Inte
rvie
w,
Obs
erva
tion,
Dia
gnos
tic T
estin
g,C
onte
nt A
naly
sis
Inte
rvie
w
Inte
rvie
w,
Obs
erva
tion
Inte
rvie
w O
bser
vatio
nD
iagn
ostic
Tes
ting
Surv
ey,
Inte
rvie
w, S
urve
y,C
onte
nt A
naly
sis
IVer
view
,Obs
erva
tion
Dia
gnos
tic T
estin
g,Su
rvey
, Con
tent
Ana
lysi
s
41
14
Dat
a C
olle
ctio
n in
Per
form
ance
/Fro
nt-E
ndA
naly
sis
To
answ
er th
is s
ues
tion
Use
thes
e so
urce
sA
nd th
ese
stra
te ie
s
Wha
t are
the
alte
rnat
ive
solu
tions
?
Wha
t is
the
cost
of
not s
olvi
ng th
epr
oble
m?
Inst
ruct
iona
l Tec
hnol
ogis
t, O
ther
Exp
erts
Inte
rvie
w, S
urve
y
Clie
nt, S
uper
viso
rO
utpu
t/Pro
duct
ivity
Rec
ords
,R
aw D
ata
(e.g
., lo
ss o
f re
venu
e)
Inte
rvie
w, C
onte
nt A
naly
sis
Oth
er A
naly
sis
Wha
t are
the
cost
s of
dif
fere
nt s
olut
ions
?In
stru
ctio
nal T
echn
olog
ist,
Oth
erE
xper
tsIn
terv
iew
Wha
t is
the
optim
um s
olut
ion?
Inst
ruct
iona
l Tec
hnol
ogis
t, O
ther
Exp
erts
Inte
rvie
w
43
4215
Wha
t To
Do
if a
Tra
inin
g N
eed
is N
ot C
onfi
rmed
Dir
ectio
ns:
Thi
s pr
oced
ural
che
cklis
t exp
lain
s, in
a s
tep-
by-s
tep
fash
ion,
wha
t to
do w
hen
a re
ques
t for
the
deve
lopm
ent o
f tr
aini
ng is
not
conf
irm
ed b
y th
e da
ta y
ou c
olle
cted
. Fol
low
the
actio
n st
eps
to id
entif
y an
d ju
stif
y no
ntra
inin
g so
lutio
ns in
a c
onst
ruct
ive
fash
ion.
--:
1. R
epor
t the
pos
sibi
lity
of u
ncon
firm
ed tr
aini
ng to
you
r su
perv
isor
as
soon
as
you
are
reas
onab
ly s
ure.
Hav
e da
ta to
sup
port
your
pos
ition
. see
k th
eir
advi
se o
n w
hat t
o do
nex
t.
2. J
oint
ly d
eter
min
e if
mor
e da
ta is
nee
ded.
3. M
aint
ain
your
obj
ectiv
ity. B
e ca
refu
l not
to le
t you
r pr
elim
inar
y co
nclu
sion
s bi
as y
our
data
col
lect
ion
activ
ity.
4. A
naly
ze y
our
data
and
wri
te a
Per
form
ance
/Fro
nt-E
nd A
naly
sis
Rep
ort.
( se
e pg
12)
5. C
lear
ly a
naly
ze a
nd r
epor
t the
initi
ator
s re
ason
s fo
r th
e tr
aini
ng r
eque
st.(
see
pg 6
) L
ist a
ll or
igin
al p
robl
em in
dica
tors
.
6. S
tate
the
perf
orm
ance
def
icit
reve
aled
by
your
ana
lysi
s. I
ndic
ate
the
idea
l ver
ses
actu
al p
erfo
rman
ce. G
ive
rele
vant
dat
a.
7. S
tate
you
r hy
poth
esis
abo
ut th
e pr
imar
y ca
use
of th
e pe
rfor
man
ce d
efic
it. G
ive
rele
vant
dat
a.
8. L
ist a
ltern
ativ
e so
lutio
ns th
at m
ight
be
cons
ider
ed a
nd w
hy.
9. C
ompa
re th
e co
sts
of tr
aini
ng v
erse
s no
n-tr
aini
ng s
olut
ions
and
the
risk
s in
volv
ed in
eac
h.
10. D
eter
min
e th
e be
st m
etho
d of
pre
sent
ing
the
data
to y
our
cust
omer
.
4445
16
Form
ativ
e C
rite
ria
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
Eva
luat
ion
Que
stio
ns
Com
preh
ensi
ve p
lann
ing
Rea
listic
pla
nnin
g
Cle
ar d
efin
ition
of
the
prob
lem
Rel
evan
t def
icits
Com
preh
ensi
veso
lutio
ns
Doe
s th
e pl
an f
or th
e pe
rfor
man
cean
alys
is s
peci
fy th
e ra
tiona
le, t
heta
rget
gro
up, d
ata
requ
irem
ents
, and
step
-by-
step
act
iviti
es?
Doe
s th
e pl
an f
or p
erfo
rman
cean
alys
is ta
ke in
to c
onsi
dera
tion
the
natu
re a
nd th
e sc
ope
of th
e ap
pare
ntpr
oble
m a
nd th
e or
gani
zatio
nal,
pers
onne
l, tim
e, a
nd r
esou
rce
cons
trai
nts?
Is .t
he p
erfo
rman
ce p
robl
em c
lear
lyde
fine
d in
term
s of
rel
iabl
e an
d va
lidda
ta o
n th
e id
eal a
nd a
ctua
lou
tcom
es (
or p
erfo
rman
ces)
?
Are
all
defi
cits
that
con
trib
ute
to th
epe
rfor
man
ce p
robl
em c
lear
lyid
entif
ied
and
clas
sifi
ed?
Rat
ing
and
Com
men
tsL
OW
HIG
H1
23
45
Hav
e th
e da
ta b
een
revi
ewed
by
the
peop
le w
ho u
nder
stan
d th
e pr
oble
man
d ca
n re
com
men
d so
lutio
ns?
Hav
e al
l rea
sona
ble
solu
tions
bee
n in
vest
igat
ed?
.140
,1"g
t.:
Per
form
ance
Ana
lysi
sE
valu
atio
n
Rea
listic
Cos
tsA
ppro
pria
te R
ecom
men
datio
nsFo
llow
-Up
Feas
ibili
ty
PR
OF
ITS
Sst
'
Z;::
:Nre
at'k
:C."
`
1: iii
4849
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
Eva
luat
ion
Form
ativ
e C
rite
ria
Que
stio
nsR
atin
g an
d C
omm
ents
LO
WH
IGH
12
34
5
Rea
listic
cos
tsA
re a
ll th
e co
sts
and
risk
s of
alte
rnat
ive
solu
tions
(an
d of
igno
ring
the
perf
orm
ance
pro
blem
) id
entif
ied
and
accu
rate
ly e
stim
ated
?
App
ropr
iate
Is th
e fi
nal s
olut
ion
(or
set o
fre
com
men
datio
nsso
lutio
ns)
sele
cted
thro
ugh
asy
stem
atic
com
pari
son
of th
e co
sts
and
bene
fits
of
all v
iabl
e al
tern
ativ
eso
lutio
ns?
Follo
w-u
p fe
asib
ility
50
Do
suff
icie
nt r
esou
rces
exi
st to
wor
kon
the
solu
tion
to th
e pr
oble
m?
Doe
s th
e Pe
rfor
man
ce/F
ront
-End
Ana
lysi
sre
port
con
tain
suf
fici
ent i
nfor
mat
ion
on w
hich
a d
ecis
ion
for
proj
ect
cont
inua
tion
can
be m
ade?
51
18
Perf
orm
ance
/Fro
nt-E
nd A
naly
sis
Rep
ort
Purp
ose:
To
prep
are
a re
port
for
com
mun
icat
ing
the
find
ings
and
rec
omm
enda
tions
of
a Pe
rfor
man
ce/F
ront
-End
Ana
lysi
s to
you
rm
anag
emen
t or
clie
nt. T
he r
epor
t sho
uld
incl
ude
the
follo
win
g in
form
atio
n.
I. P
roje
ct I
nitia
tion
Who
mad
e th
e re
ques
t for
the
trai
ning
? C
omm
ent:
Wha
t Org
aniz
atio
n w
here
they
fro
m, l
ocat
ion?
Com
men
t:
Who
are
the
perf
orm
ers?
Com
men
t:
Wha
t are
the
job
title
s, jo
b de
scri
ptio
ns?
Com
men
t:
Wha
t was
the
reas
on g
iven
for
the
requ
est a
nd w
hat e
vide
nce
or d
ata
was
giv
en?
Com
men
t:
II. D
ata
Col
lect
ion
and
Ana
lysi
s
Wha
t wer
e th
e m
etho
ds a
nd in
stru
men
ts u
sed
to c
olle
ct th
e da
ta?
Com
men
t:
Doe
s th
e da
ta c
onfi
rm th
e pr
oble
m a
s re
port
ed to
you
? C
omm
ent:
Wha
t is
the
gap
in p
erfo
rman
ce d
esir
ed v
erse
s ac
tual
? C
omm
ent:
53
5219
Wha
t is
the
prim
ary
and
othe
r ca
uses
of
the
defi
cit?
Com
men
t:
III.
Con
clus
ions
Wha
t sol
utio
ns a
re f
easi
ble?
Com
men
t:
Wha
t is
the
cost
of
igno
ring
the
prob
lem
? C
omm
ent:
Wha
t is
the
cost
of
the
solu
tions
? C
omm
ent:
Wha
t is
the
optim
al s
olut
ion?
Com
men
t:
5 4
OY
h
5:FO
NM
AM
MW
OM
O*
agnI
AM
W;W
WW
W=
Sige
N
Tra
inin
gN
eed?
Wha
t wou
ld h
appe
n if
you
left
it a
lone
?
BE
ST C
OPY
AV
AIL
AB
LE
57
TR
AIN
ING
NE
ED
S A
NA
LY
SIS
In tr
aini
ng n
eeds
ana
lysi
s, th
e D
evel
oper
/Tea
cher
is s
till c
once
rned
with
the
adeq
uacy
of
the
data
colle
ctio
n pl
an, a
nd w
ith
the
valid
ity o
f th
e an
alys
is it
self
and
the
type
s of
ana
lytic
al to
ols
used
. Thi
s is
nec
essa
ry to
conc
lude
that
the
skill
& k
now
ledg
e
anal
ysis
and
cla
ssif
icat
ion
are
perf
orm
ed c
orre
ctly
. The
Dev
elop
er/T
each
er o
vers
ees
the
plan
ning
and
mak
es s
ure
the
reso
urce
s
requ
ired
and
dat
a ar
e av
aila
ble
to th
e te
am, h
e al
so c
oord
inat
es w
ith o
ther
gro
ups.
The
goa
l is
topr
oduc
e a
deta
iled
rele
vant
ski
ll
& k
now
ledg
e lis
t whi
ch m
akes
up
the
trai
ning
and
a p
relim
inar
y de
scri
ptio
n of
the
mos
tef
fect
ive
trai
ning
del
iver
y st
rate
gy o
r
reco
mm
enda
tion
for
othe
r so
lutio
ns. T
he D
evel
oper
/Tea
cher
aga
in is
in a
pos
ition
to r
evie
w th
e ou
tcom
esof
the
anal
ysis
for
com
plet
enes
s.
The
man
ager
mus
t ass
ure
the
follo
win
g qu
estio
ns h
ave
been
ade
quat
ely
answ
ered
reg
ardi
ng le
arne
ran
d ta
sk a
naly
sis.
PLA
N
Doe
s th
e pl
an ta
ke in
to c
onsi
dera
tion
the
natu
re a
nd s
cope
of
the
proj
ect t
he r
esou
rces
,re
quir
ed a
nd th
e co
nstr
aint
s if
any
?
DA
TA A
re th
e st
rate
gies
for
col
lect
ing
all r
elev
ant d
ata
on le
arne
rs a
nd s
kill/
know
ledg
ere
quir
emen
ts o
bjec
tivel
y co
llect
ed
thro
ugh
appr
opri
ate
stra
tegi
es i.
e. in
terv
iew
, obs
erva
tion
etc.
. Is
the
data
suf
fici
ent t
oco
mpl
ete
a de
taile
d ta
sk a
naly
sis?
Has
the
5b59
21
targ
et p
opul
atio
n be
en id
entif
ied
and
anal
yzed
? H
ave
ther
e en
try
leve
l ski
ll an
d kn
owle
dge
been
det
erm
ined
?
TA
SK
Doe
s th
e sk
ill &
kno
wle
dge
list a
ppea
r co
mpl
ete,
acc
urat
e an
d su
ffic
ient
ly d
etai
led?
Doe
s th
e an
alys
t ide
ntif
y pr
ereq
uisi
tes
and
wha
t nee
ds to
be
trai
ned
? D
oes
the
fina
l ski
ll &
kno
wle
dge
list i
dent
ify
only
thos
e to
be
trai
ned?
Whe
n th
ese
ques
tions
hav
e be
en s
atis
fact
orily
ans
wer
ed w
e ar
e re
ady
for
the
next
pha
se.
PRE
LIM
INA
RY
DE
LIV
ER
Y S
TR
AT
EG
Y
In a
ssur
ing
the
deve
lope
r is
on
the
righ
t tra
ck y
ou m
ust m
onito
r th
e fo
llow
ing:
I) U
sing
dat
a co
llect
ed f
rom
the
perf
orm
ance
, lea
rner
, and
task
ana
lysi
s co
nduc
ted
earl
ier
is a
dequ
ate
cons
ider
atio
n gi
ven
to
a. W
ho a
re th
e st
uden
ts?
b. H
ow w
ill th
e in
stru
ctio
n be
dev
elop
ed, d
eliv
ered
? do
es it
exi
st n
ow?
c. W
hat a
re th
e re
sour
ces
and
sign
ific
ant c
onst
rain
ts?
d. W
here
are
the
perf
orm
ers
and
whe
re w
ill p
rogr
am ta
ke p
lace
?
b()
6122
e. W
hen
is th
e v-
form
ance
req
uire
d? W
hen
the
prog
ram
will
be
read
y, h
ow lo
ng it
will
take
, the
num
ber
and
freq
uenc
y of
offe
ring
s?
2) I
dent
ifie
d al
tern
ate
solu
tions
, ana
lyze
d th
e sa
me
for
feas
ibili
ty c
osts
and
eff
ectiv
enes
s pa
rtic
ular
ly:
a. I
mpa
ct-
Lev
el 3
and
4 e
valu
atio
n th
e bo
ttom
line
est
imat
es o
n im
prov
ed jo
b pe
rfor
man
ce p
rodu
ctiv
ity, a
dapt
abili
ty, a
nd
mot
ivat
ion
b. O
utco
mes
- L
evel
2 e
valu
atio
n di
d th
e st
uden
ts le
arn
the
obje
ctiv
e an
d in
an
effe
ctiv
e an
d ef
fici
ent m
anne
r.
c. P
roce
ss-
Perf
orm
ance
dur
ing
the
prog
ram
bot
h of
the
lear
ner
and
med
ia s
elec
ted.
Lev
el I
eva
luat
ion
.
6263
23
Che
cklis
t of
Act
iviti
es d
urin
g T
rain
ing
Nee
ds A
naly
sis
PLA
NFo
r co
nduc
ting
lear
ner
anal
ysis
.Fo
r su
itabl
e da
ta c
olle
ctio
n st
rate
gies
for
lear
ner
anal
ysis
.Fo
r co
nduc
ting
task
ana
lysi
s.Fo
r su
itabl
e da
ta c
olle
ctio
n st
rate
gies
for
task
ana
lysi
s.T
o ob
tain
coo
pera
tion
from
oth
er g
roup
s in
dat
a co
llect
ion
for
trai
ning
nee
ds a
naly
sis.
Iden
tify
Iden
tify
skill
s an
d kn
owle
dge
requ
ired
to p
erfo
rm th
e jo
b or
task
.Id
entif
y po
tent
ial d
eliv
ery
stra
tegi
es.
Suita
ble
sour
ces
of in
form
atio
n fo
r ta
sk a
naly
sis.
Mas
ter
Perf
orm
er to
dem
onst
rate
var
ious
ski
lls r
equi
red
for
perf
orm
ing
the
job
or ta
sk.
Col
lect
Col
lect
and
ana
lyze
lear
ner
anal
ysis
dat
a an
d su
mm
ariz
e th
e re
sults
.C
olle
ct a
nd a
naly
ze ta
sk a
naly
sis
data
and
sum
mar
ize
the
resu
lts.
Ana
lysi
sC
ondu
ct a
cos
t-ef
fect
iven
ess
anal
ysis
of
pote
ntia
l del
iver
y st
rate
gies
.M
ake
a pr
elim
inar
y se
lect
ion
of th
e op
timum
del
iver
y st
rate
gy.
Mon
itor
& R
evie
wT
rain
ing
need
s an
alys
is a
ctiv
ities
and
pro
vide
fee
dbac
k.R
evie
w a
nd a
ppro
ve th
e ou
tcom
es o
f tr
aini
ng n
eeds
ana
lysi
s.R
evie
w f
inal
list
of
skill
s an
d kn
owle
dge
requ
ired
for
per
form
ing
the
job
or ta
sk.
65
6424
To
answ
er th
ese
ques
tions
Wha
t tas
ks c
onst
itute
the
job?
Wha
t is
the
stru
ctur
e of
the
task
?
Dat
a C
olle
ctio
n in
Tas
k A
naly
sis
Use
thes
e so
urce
s
Job
desc
ript
ions
Doc
umen
tatio
n on
prac
tices
Wha
t are
the
skill
s an
d kn
owle
dge
requ
ired
to p
erfo
rm th
e ta
sk?
Tec
hnic
al d
ocum
enta
tion
Wha
t pre
requ
isite
ski
lls a
re n
eede
d to
Tra
inin
g m
ater
ials
perf
orm
hig
her-
leve
l ski
lls?
How
will
dif
fere
nt le
vels
of
enab
ling
Tra
inin
g m
ater
ials
obje
ctiv
es h
elp
stud
ents
ach
ieve
the
Supe
rvis
ors
over
all o
bjec
tive?
And
thes
e st
rate
gies
Con
tent
ana
lysi
s
Obs
erva
tion
Inte
rvie
w
Wha
t are
the
diff
eren
t lev
els
of s
kills
Mas
ter
perf
orm
erO
bser
vatio
nan
d kn
owle
dge,
and
how
are
they
rela
ted
to e
ach
othe
r?
Wha
t pie
ces
of in
form
atio
n ar
e to
be
pres
ente
d to
the
stud
ents
and
in w
hat
orde
r?SM
Es
Wha
t are
the
step
s in
the
task
Job
desc
ript
ions
proc
edur
e?C
onte
nt a
naly
sis
Obs
erva
tion
67
6625
To
answ
er th
ese
ques
tions
Dat
a C
olle
ctio
n in
Tas
k A
naly
sis
Use
thes
e so
urce
sA
nd th
ese
stra
tegi
es
How
is th
is ta
sk p
erfo
rmed
?
Wha
t is
the
stru
ctur
e of
the
stan
dard
proc
edur
e?
Wha
t are
the
actio
n an
d de
cisi
on s
teps
in th
e pr
oced
ure?
Wha
t is
the
algo
rith
m f
or s
olvi
ng th
epr
oble
m?
Wha
t is
the
trig
geri
ng e
vent
?
Wha
t are
the
nece
ssar
y in
puts
?
Wha
t too
ls a
re r
equi
red?
Wha
t are
the
outp
uts?
Wha
t are
the
stan
dard
s fo
r ac
cept
able
perf
orm
ance
?
Wha
t is
the
freq
uenc
y of
the
subt
ask
inth
e w
orkp
lace
?
68
Doc
umen
tatio
n on
prac
tices
Tec
hnic
al d
ocum
enta
tion
Tra
inin
g m
ater
ials
Tec
hnic
al d
ocum
enta
tion
Man
ager
s, S
uper
viso
rs
Mas
ter
perf
orm
ers
Job
desc
ript
ions
Inte
rvie
wO
bser
vatio
n
Flow
char
ting
Flow
char
ting
Inte
rvie
w
Con
tent
ana
lysi
sO
bser
vatio
n
Doc
umen
tatio
n on
pra
ctic
esIn
terv
iew
Tec
hnic
al d
ocum
enta
tion
SME
's. M
aste
r Pe
rfor
mer
sIn
terv
iew
6926
To
answ
er th
ese
ques
tions
Dat
a C
olle
ctio
n in
Tas
k A
naly
sis
Use
thes
e so
urce
sA
nd th
ese
stra
tegi
es
How
muc
h tim
e is
nee
ded
to c
ompl
ete
Supe
rvis
ors,
Mas
ter
Perf
orm
ers
the
subt
ask?
How
dif
ficu
lt is
the
subt
ask
to p
erfo
rm?
How
dif
ficu
lt is
the
subt
ask
to le
arn?
Are
ther
e an
y da
nger
ous
cons
eque
nces
of in
corr
ect p
erfo
rman
ce?
SME
's, M
aste
r Pe
rfom
'ers
SME
's, M
aste
r Pe
rfor
mer
s
71
27
To
answ
er th
ese
ques
tions
Dat
a C
olle
ctio
n in
Lea
rner
Ana
lysi
s
Use
thes
e so
urce
sA
nd th
ese
stra
tegi
es
Wha
t are
the
jobs
(or
job
title
s)of
the
targ
et p
opul
atio
n?
Wha
t are
the
jobs
of
the
seco
ndar
ypo
pula
tions
?
How
man
y pe
ople
are
em
ploy
ed in
each
job?
Wha
t are
the
dem
ogra
phic
char
acte
rist
ics
of th
e st
uden
ts (
e.g.
,ag
e, s
ex, a
nd le
ngth
of
serv
ice)
?
How
muc
h su
ppor
t will
the
stud
ents
rece
ive
from
thei
r bo
sses
and
pee
rsfo
r at
tend
ing
this
trai
ning
?
Wha
t is
the
empl
oym
ent l
evel
of
the
stud
ent a
s re
late
d to
inst
ruct
or?
Wha
t rel
evan
t job
exp
erie
nces
do
the
stud
ents
hav
e?
Wha
t rel
evan
t tra
inin
g ha
ve th
ey h
ad?
Wha
t is
thei
r ge
nera
l edu
catio
nal
back
grou
nd"
Cor
pora
te c
ensu
s
Job
desc
ript
ion
Job
hist
ory
Gen
eral
per
sonn
elre
cord
s
Man
agem
ent
Supe
rvis
ors
Rep
rese
ntat
ive
stud
ents
Rep
rese
ntat
ive
stud
ents
Man
agem
ent
Supe
rvis
ors
Form
er in
stru
ctor
s
Rep
rese
ntat
ive
stud
ents
Con
tent
ana
lysi
s of
reco
rds
and
repo
rts
Inte
rvie
wSu
rvey
Con
tent
ana
lysi
s of
reco
rds
and
repo
rts
Inte
rvie
w
Surv
eyD
iagn
ostic
test
ing
7372
28
To
answ
er th
ese
ques
tions
Dat
a C
olle
ctio
n in
Lea
rner
Ana
lysi
s
Use
thes
e so
urce
sA
nd th
ese
stra
tegi
es
Wha
t ski
lls d
o th
ey a
lrea
dy h
ave?
Wha
t cur
rent
pra
ctic
es a
nd h
abits
have
to b
e el
imin
ated
?
Wha
t inf
orm
atio
n an
d kn
owle
dge
doth
ey h
ave
abou
t the
task
?
Wha
t mis
conc
eptio
ns a
re th
ey li
kely
toha
ve?
Perf
orm
ance
Ana
lysi
s R
epor
tO
bser
vatio
nD
iagn
ostic
test
ing
Job
desc
ript
ion
Tra
inin
g re
cord
s
Aca
dem
ic r
ecor
dsC
orpo
rate
test
s
Wha
t is
thei
r at
titud
e to
war
d th
etr
aini
ng to
pic?
Wha
t is
thei
r at
titud
e to
war
d th
ede
liver
y st
rate
gy?
Wha
t is
thei
r le
vel o
f la
ngua
ge?
Wha
t are
thei
r la
ngua
ge p
refe
renc
es?
Wha
t are
thei
r le
arni
ng s
kills
?
Wha
t typ
es o
f le
arni
ng a
ctiv
ities
do
they
pre
fer?
75
7429
.44,
"WW
2
:555
5555
aeA
0
Tra
inin
gN
eeds
Ana
lysi
sE
valu
atio
n
* C
ompr
ehen
sive
pla
nnin
g*
Rea
listic
Pla
nnin
gD
ata
colle
ctio
nR
elev
ant
Suff
icie
nt
* L
earn
er a
naly
sis
data
Skill
/Kno
wle
dge
Req
uire
d
Skill
/Kno
wle
dge
Val
idity
*Rel
evan
t dat
a fo
r D
eliv
ery
Stra
tegy
Es
Fs
Aff
ne.4
,g,
-10,
1$"9
,11r
et.
:f1
f: 4
4141
:5w
43
7 f;
77
Form
ativ
e C
rite
ria
Com
preh
ensi
ve p
lann
ing
Rea
listic
pla
nnin
g
Rel
evan
t dat
a
Dat
a co
llect
ion
Tra
inin
g N
eeds
Ana
lysi
s E
valu
atio
n
Que
stio
nsR
atin
g an
d C
omm
ents
Doe
s th
e pl
an id
entif
y ea
ch s
tep
inth
is p
hase
and
spe
cify
the
pers
onne
lan
d sc
hedu
le r
equi
rem
ents
?
Doe
s th
e pl
an ta
ke in
to c
onsi
dera
tion
the
natu
re a
nd th
e sc
ope
of th
epr
ojec
t, th
e re
sour
ces,
and
the
cons
trai
nts?
Are
all
rele
vant
dat
a on
lear
ner
char
acte
rist
ics
and
skill
/kno
wle
dge
requ
irem
ents
obj
ectiv
ely
colle
cted
thro
ugh
appr
opri
ate
stra
tegi
es?
Hav
e ap
prop
riat
e da
ta c
olle
ctio
nte
chni
ques
bee
n id
entif
ied?
Suff
icie
nt d
ata
Has
suf
fici
ent d
ata
been
col
lect
ed to
com
plet
e a
deta
iled
task
ana
lysi
s?
Lea
rner
ana
lysi
s da
ta
78
Has
the
targ
et p
opul
atio
n be
enid
entif
ied
and
has
enou
gh d
etai
lbe
en c
olle
cted
abo
ut th
eir
entr
y le
vel
skill
s an
d kn
owle
dge?
LO
WH
IGH
12
34
5
79
30
Form
ativ
e C
rite
ria
Tra
inin
g N
eeds
Ana
lysi
s E
valu
atio
n
Que
stio
ns
Skill
/kno
wle
dge
requ
irem
ents
Alte
rnat
ives
to tr
aini
ng
Skill
/kno
wle
dge
valid
ity
Rel
evan
t dat
a fo
rde
liver
y st
rate
gy
60
Is th
e sk
ill k
now
ledg
e lis
t com
plet
e,ac
cura
te, a
nd in
suf
fici
ent d
etai
l?
Has
eac
h sk
ill o
r kn
owle
dge
been
anal
yzed
for
alte
rnat
ives
to tr
aini
ng?
Rat
ing
and
Com
men
tsL
OW
HIG
H1
23
45
Has
eac
h sk
ill a
nd k
now
ledg
e id
entif
ied
as"e
ntry
leve
l" o
r "t
o be
trai
ned"
? D
oes
the
fina
lsk
ill/k
now
ledg
e lis
t inc
lude
onl
yth
ose
whi
ch a
re to
be
trai
ned?
Are
all
rele
vant
cos
t and
ben
efit
data
for
sele
ctin
g pr
elim
inar
y de
liver
y st
rate
gyob
ject
ivel
y co
llect
ed th
roug
h ap
prop
riat
e st
rate
gies
81
U
Tas
k D
etai
lsU
sers
:C
ours
e In
stru
ctor
, Tea
cher
, Dev
elop
er
Purp
ose:
To
iden
tify
and
reco
rd d
etai
ls o
f a
task
, a li
st o
f su
btas
ks. a
nd in
form
atio
n so
urce
s fo
r ea
ch s
ubta
sk.
Inpu
ts:
Dat
a fr
om ta
sk a
naly
sis
inte
rvie
ws,
con
tent
ana
lyse
s, a
nd o
bser
vatio
ns.
Dir
ectio
ns:
Thi
s w
orks
heet
con
sist
s of
two
faci
ng p
ages
.O
n th
e fi
rst p
age,
iden
tify
the
cour
se a
nd th
e ta
sk.
Wri
te a
list
of
subt
asks
. For
eac
h su
btas
k, b
rief
ly n
ote
your
sou
rce
of in
form
atio
n (i
nfor
man
t or
docu
men
t) f
or la
ter
veri
fica
tion
On
the
seco
nd p
age,
not
e th
e de
tails
of
the
task
in th
e ap
prop
riat
e lin
e. Y
ou d
o no
t hav
e to
com
plet
e al
l det
ails
.If
som
e im
port
ant t
ask
deta
il do
es n
ot f
it in
any
of
the
cate
gori
es in
this
pag
e, r
ecor
d it
unde
r "A
dditi
onal
Inf
orm
atio
n".
It is
hel
pful
to e
stab
lish
a nu
mbe
ring
sys
tem
for
task
s an
d su
btas
ks.
You
may
nee
d m
ore
than
one
pag
e to
list
all
asso
ciat
ed ta
sks.
Rel
ated
Wor
kshe
ets:
Skill
Kno
wle
dge
Req
uire
men
ts1
ise
the
task
det
ails
fro
m th
is w
orks
heet
to d
eriv
e a
com
plet
e se
t of
skill
/kno
wle
dge
requ
irem
ents
.
8382
32
(Pag
e 1)
Cou
rse:
Tas
k #:
Des
crip
tion:
Tas
k D
etai
ls
Subt
asks
Des
crip
tion
Sour
ces
of I
nfor
mat
ion
8
8 4
13
Tas
k D
etai
ls
Tri
gger
ing
even
t:
Inpu
t:
Too
ls a
nd e
quip
men
t:
Out
put:
Stan
dard
s:
ITtm
e re
quir
ed:
[Im
port
ance
:
Freq
uenc
y:
Perf
orm
ance
dif
ficu
lty:
1Lea
rnin
g di
ffic
ulty
:
War
ning
s: 687
34
How
To
Der
ive
Skill
Kno
wle
dge
Req
uire
men
ts
].M
ake
a lis
t of
skill
s an
d kn
owle
dge
for
perf
orm
ing
the
job
or th
e ta
sks
base
d on
you
r ta
sk a
naly
sis.
2. T
he s
ame
skill
may
be
invo
lved
in m
ore
than
one
task
. Rem
ove
dupl
icat
e st
atem
ents
of
such
gen
eral
ski
lls.
3. K
eep
the
stat
emen
ts o
f sk
ills
and
know
ledg
e at
the
sam
e le
vel o
f sp
ecif
icity
. Don
't m
ake
som
e ve
ry g
ener
al a
nd o
ther
s ve
ry
spec
ific
.
4. A
sk th
e In
stru
ctio
nal T
echn
olog
ist a
nd a
n SM
E to
rev
iew
you
r lis
t of
skill
s an
d kn
owle
dge
and
mak
e su
itabl
e re
visi
ons.
5. T
rans
fer
your
list
to th
e Sk
ills/
Kno
wle
dge
Req
uire
men
ts W
orks
heet
13)
.
6. C
ross
-ref
eren
ce s
kills
and
kno
wle
dge
Item
s to
ass
ocia
ted
task
s an
d su
btas
ks.
7. U
sing
dat
a fr
om y
our
lear
ner
anal
ysis
, che
ck o
ff th
ose
skill
s an
d kn
owle
dge
your
targ
et s
tude
nts
alre
ady
poss
ess.
8. R
evie
w th
e re
mai
ning
ski
lls a
nd k
now
] dg
e an
d ch
eck
off
thos
e w
hich
can
be
prov
ided
thro
ugh
docu
men
tatio
n or
job
aids
.
Her
e ar
e th
e fa
ctor
s w
hich
sug
gest
doc
umen
tatio
n or
job
aids
:
9 R
evie
w th
e do
cum
enta
tion
on jo
b ai
d ite
ms
to s
ee if
any
of
them
req
uire
add
ition
al tr
aini
ng to
sup
port
thei
r us
e. C
heck
trai
ning
for
thes
e ite
ms
(in
addi
tion
to d
ocum
enta
tion/
job
aids
).
Tas
ks w
hich
do
not h
ave
to b
e pe
rfor
med
rap
idly
Tas
ks w
ith m
ajor
neg
ativ
e co
nseq
uenc
es f
or in
corr
ect p
erfo
rman
ce
88
8815
Seve
ral u
nrel
ated
pie
ces
of in
form
atio
n w
hich
nee
d no
t be
mem
oriz
ed
Tas
ks w
hich
will
cha
nge
in th
e ne
ar f
utur
e
Len
gthy
task
s w
ith s
ever
al s
teps
10. C
heck
the
rem
aini
ng J
obs
for
trai
ning
.
11. A
sk th
e In
stru
ctio
nal T
echn
olog
ist a
nd a
n :W
E to
rev
iew
you
r w
orks
heet
and
mak
e su
itabl
e re
visi
ons.
'00
9136
Skill
/Kno
wle
dze
Req
uire
men
ts
Skill
/Kno
wle
dge
Tas
k/Su
btas
kD
escr
iptio
nE
ntry
Doc
/JA
Tra
in
9 2
9317
Shor
tcut
s
Tra
inin
g N
eeds
Ana
lysi
s
1.L
imit
data
col
lect
ion
activ
ities
: Use
sm
alle
r da
ta b
ase
and
SME
s. R
ely
entir
ely
on f
aste
r da
ta c
olle
ctio
n ac
tiviti
es, e
.g.,
cont
ent
anal
ysis
.
2.In
form
ally
lim
it th
e sc
ope
of th
e pr
ojec
t, by
elim
inat
ing
less
cri
tical
are
asof
the
job
from
the
task
ana
lysi
s, e
.g.,
task
s do
ne le
ssfr
eque
ntly
, tas
ks f
ewer
peo
ple
have
trou
ble
with
, and
task
s w
hich
cau
se n
ore
al p
robl
em if
don
e in
corr
ectly
.
3.D
o no
t per
form
dia
gnos
tic te
stin
g as
a p
art o
f le
arne
ran
alys
is.
4.Ig
nore
sec
onda
ry p
opul
atio
ns
9438
INST
RU
CT
ION
AL
DE
SIG
N
The
ana
lysi
s w
e ju
st c
ompl
eted
can
be
used
as
a bl
uepr
int t
o cr
eate
Tra
inin
g. I
n T
rain
ing
need
s an
alys
is w
e ar
e st
ill
conc
erne
d w
ith d
ata
colle
ctio
n an
d an
alys
is. A
lthou
gh it
is te
mpt
ing
to s
tart
dev
elop
ing
mat
eria
ls y
ou p
ay f
or it
in th
e lo
ng r
un b
y
incl
udin
g ir
rele
vant
con
tent
and
exc
essi
ve r
evis
ions
that
pro
per
inst
ruct
iona
l des
ign
wor
k w
ould
rav
e el
imin
ated
.
The
maj
or o
utco
mes
of
this
piv
otal
pha
se in
clud
e : d
eter
min
ing
inst
ruct
iona
l obj
ectiv
es, d
evel
opin
g te
sts
for
the
obje
ctiv
es,
conf
irm
ing
over
all d
eliv
ery
stra
tegy
and
dev
elop
ing
desi
gn s
peci
fica
tions
to b
e us
ed in
the
next
pha
se m
ater
ial d
evel
opm
ent.
The
man
ager
pla
ys a
key
rol
e in
all
com
pone
nts
but p
erha
ps th
e m
ost c
ritic
al is
in th
e de
term
inat
ion
of in
stru
ctio
nal o
bjec
tives
and
inte
ract
ion
with
the
clie
nt is
impe
rativ
e in
ord
er to
ass
ure
that
: 1)
The
obj
ectiv
es a
re a
ppro
pria
te to
mee
t the
clie
nts
need
. 2)
The
prog
ram
is d
esig
ned
to m
eet t
he o
bjec
tive.
Tra
inin
g th
at is
not
ful
ly c
olla
bora
tive
is li
kely
to b
e st
erile
, and
inef
fect
ive.
Bef
ore
the
Des
igne
r/T
each
er g
ives
app
rova
l the
obj
ectiv
e st
atem
ent s
houl
d be
com
plet
ed to
his
sat
isfa
ctio
n. T
he o
bjec
tive
Stat
emen
t sho
uld
incl
ude:
1. T
he n
ame
of th
e re
spon
sibl
e cl
ient
and
his
fun
ctio
n. 2
. The
pre
sent
situ
atio
n an
d ne
ed f
or tr
aini
ng a
ctio
n 3.
The
trai
ning
obj
ectiv
es. 4
. The
trai
ning
act
ions
pro
pose
d w
ith d
ates
5. T
he e
stim
ated
cos
ts a
nd b
enef
its. 6
. The
trai
ning
res
ourc
es
requ
ired
.
The
Des
igne
r/T
each
er u
tiliz
es th
e st
atem
ent i
n th
e fo
llow
ing:
1. G
rant
app
rova
l of
the
actio
ns to
be
take
n by
his
trai
ning
pers
onne
l. 2.
Gai
n co
nfid
ence
that
it c
ontr
ibut
es to
rea
l cor
pora
te o
bjec
tives
. 3. C
onfi
denc
e th
at li
ne m
anag
emen
t are
in a
pos
ition
to im
plem
ent w
hat i
s go
ing
to b
e tr
aine
d. 4
. A b
asis
for
con
trol
and
ass
istin
g hi
s st
aff
duri
ng th
e pr
ojec
t. M
anag
emen
t can
not
mon
itor
proj
ects
with
out r
elev
ant p
oint
s of
ref
eren
ce a
nd a
dequ
ate
data
. The
obj
ectiv
e st
atem
ents
pro
vide
s bo
th.
97
9639
DE
TE
RM
INE
IN
STR
UC
TIO
NA
L O
BJE
CT
IVE
S
Idea
l obj
ectiv
es w
ill s
peci
fy w
hat t
he le
arne
r w
ill b
e ab
le to
do
at th
e en
d (t
erm
inal
)th
at h
e w
as n
ot a
ble
to a
t the
beg
inni
ng
of th
e pr
ogra
m. T
est o
r te
st it
ems
(ena
blin
g) w
ill s
how
that
he
can
do it
. The
refo
re in
mon
itori
ng o
bjec
tives
the
man
ager
mus
t
insu
re th
at I
ndiv
idua
l Obj
ectiv
es in
clud
e: 1
. Beh
avio
r- T
he o
bjec
tive
clea
rly
iden
tify
obse
rvab
le, m
easu
rabl
e, r
epea
tabl
e be
havi
or
whi
ch th
e st
uden
t can
dem
onst
rate
.
2. S
peci
fies
the
cond
ition
s un
der
whi
ch it
the
perf
orm
ance
is to
take
pla
ce.
3. S
ets
the
stan
dard
of
perf
orm
ance
in te
rms
of ti
me
or e
rror
lim
its in
whi
ch m
inim
al a
ccep
tabl
e pe
rfor
man
ce ta
kes
plac
e or
how
we
will
det
erm
ine
whe
n m
aste
ry is
ach
ieve
d. H
e m
ust a
lso
look
at a
ll ob
ject
ives
for
:
1. R
elev
ance
- A
re th
e en
ablin
g ob
ject
ives
rel
evan
t to
achi
evem
ent o
f th
e te
rmin
al o
bjec
tive
and
Are
the
term
inal
obj
ectiv
es
rele
vant
to a
chie
vem
ent o
f th
e de
sire
d pe
rfor
man
ce?
2. S
uffi
cien
t- D
o th
e en
ablin
g ob
ject
ives
spe
cify
all
the
requ
irem
ents
whi
ch s
um to
the
perf
orm
ance
req
uire
d in
the
term
inal
obje
ctiv
es?
3. A
ppro
pria
te-
Is th
e la
ngua
ge s
cope
and
dif
ficu
lty o
f th
e se
t of
obje
ctiv
es s
uita
ble
to th
e ta
rget
aud
ienc
e?
TE
STS
AN
D T
EST
SPE
CIF
ICA
TIO
NS
Tes
t spe
cifi
catio
ns n
eed
to b
e ap
prov
ed b
y th
e D
esig
ner/
Tea
cher
. The
y sh
ould
incl
ude:
the
skill
s/kn
owle
dge
cove
red
,
inst
ruct
iona
l obj
ectiv
es, i
nstr
uctio
nal c
onte
nt c
over
ed b
y th
e te
st, a
s w
ell a
s, th
e ty
pe o
f te
st it
em a
nd te
st. I
n de
term
inin
g th
ese
the
next
two
char
ts s
houl
d be
use
d.
99
98
40
SEL
EC
TIO
N O
F IN
STR
UC
TIO
NA
L M
ED
IA A
ND
ME
TH
OD
S
In s
elec
tion
of m
edia
the
firs
t thi
ng th
e de
sign
ers
shou
ld h
ave
cons
ider
ed a
re,
attr
ibut
es r
equi
red
by th
e ob
ject
ives
and
thei
r do
mai
n of
lear
ning
: au
dito
ry, v
isua
l or
kine
sthe
tic a
nd th
at r
equi
red
by th
e le
arne
rs e
.g. r
eadi
ng le
vel,
cogn
itive
abi
lity.
Con
side
ratio
n th
en m
ust b
e gi
ven
to c
osts
and
ava
ilabi
lity
of th
e pr
efer
red
med
ia. F
or th
is r
easo
n, it
is b
est t
o ha
ve d
esig
ners
subm
it a
list o
f th
e re
quir
ed a
ttrib
utes
alo
ng w
ith a
alte
rnat
e m
etho
d if
one
exi
sts.
If d
eliv
ery
is to
be
grou
ped
pace
d an
d in
stru
ctor
s ar
e av
aila
ble
then
the
man
ager
s fi
rst c
hoic
e is
usu
ally
lect
ure
and
disc
ussi
on a
s th
is is
eas
iest
on
his
staf
f an
d ca
n be
eff
ectiv
e if
lear
ners
are
hig
hly
mot
ivat
ed a
nd e
xper
ienc
ed a
t tur
ning
kno
wle
dge
into
app
licat
ion.
Mor
e im
port
antly
thou
gh f
or m
ost t
rain
ing
is a
ssur
ance
of
tran
sfer
of
lear
ning
. Alth
ough
this
is m
ore
diff
icul
t for
the
inst
ruct
or (
faci
litat
ion
skill
is n
eces
sary
) be
havi
oral
bas
ed in
stru
ctio
n is
mor
e ef
fect
ive
with
mos
t lea
rner
s. T
he c
lose
r to
rea
l
life
prac
tice
the
case
stu
dy o
r si
mul
atio
n gi
ven
an e
ffec
tive
sum
mar
y is
pra
ctic
ed th
e gr
eate
r th
e ch
ance
for
tran
sfer
. If
ther
e is
oppo
rtun
ity f
or le
arne
r co
ntro
lled
inst
ruct
ion
then
var
ious
pro
gram
med
inst
ruct
ion
tech
niqu
es s
houl
d be
con
side
red:
sel
f-pa
ced
CB
T, i
nter
activ
e-vi
deo,
aud
io a
nd/o
r vi
deo.
Whi
chev
er c
hoic
es a
re m
ade
depe
ndin
g on
whi
ch c
hara
cter
istic
s fi
t the
situ
atio
n, m
anag
ers
mus
t sig
n of
f on
all
cont
ent o
f
maj
or le
arni
ng a
ctiv
ities
for
the
even
ts o
f le
arni
ng: m
otiv
atio
n, o
rien
tatio
n, p
erqu
isite
s, p
rese
ntat
ion,
lear
ning
gui
danc
e, p
ract
ice,
feed
back
, ass
essm
ent,
rete
ntio
n an
d tr
ansf
er r
egar
dles
s of
med
ia o
r m
etho
d.
1 01
10
41
Bef
ore
proc
eedi
ng to
mat
eria
l dev
elop
men
t the
man
ager
mus
t be
satis
fied
that
: the
med
ia s
elec
ted
are
appr
opri
ate
for
the
obje
ctiv
es, t
he le
arne
r, th
e de
sign
and
the
deliv
ery
fact
ors.
Tha
t the
lear
ning
act
iviti
es a
re b
ased
on
the
even
ts o
f in
stru
ctio
n an
d
are
cons
iste
nt w
ith th
e ob
ject
ives
.
103
102
42
Che
cklis
t of
Act
iviti
es D
urin
g In
stru
ctio
nal D
esig
n
PLA
N Inst
ruct
iona
l obj
ectiv
es f
or th
e co
urse
.,
Li
Cou
rse
stru
ctur
e.D
esig
n sp
ecif
icat
ions
for
the
cour
se.
Spec
ific
pre
sent
atio
n te
chni
ques
for
eac
h in
stru
ctio
nal o
bjec
tive.
Spec
ify
lear
ning
act
iviti
es f
or e
ach
inst
ruct
iona
l obj
ectiv
e.
Con
firm
The
cho
ice
of th
e de
liver
y st
rate
gy.
Dev
elop
II
Cri
teri
on te
st.
Coo
rdin
ate
Ong
oing
eva
luat
ion
of v
ario
us p
rodu
cts
duri
ng th
is p
hase
.
105
104
43
Dev
elo
inIn
stru
ctio
nal O
bjec
tives
Purp
ose:
To
spec
ify
inst
ruct
iona
l obj
ectiv
es f
or a
cou
rse
and
to r
evie
w th
em.
Inpu
t:Sk
ill/k
now
ledg
e re
quir
emen
ts a
nd ta
sk d
etai
ls f
rom
trai
ning
nee
ds a
naly
sis.
Dir
ectio
ns:
1.Fi
ll ou
t a w
orks
heet
for
eac
h si
gnif
ican
t ski
ll or
kno
wle
dge
iden
tifie
d du
ring
gai
ning
nee
ds a
naly
sis
whi
ch is
cla
ssif
ied
unde
rtr
aini
ng.
2.Y
ou d
o no
t hav
e to
fill
in e
very
box
in th
is w
orks
heet
. Do
not c
lutte
r it
up w
ith o
bvio
us, u
nnec
essa
ry, o
r tr
ivia
l inf
orm
atio
n.3.
Prov
ide
info
rmat
ion
in th
e be
havi
or b
ox to
cle
arly
indi
cate
wha
t is
bein
g do
ne.
4.L
ist o
nly
criti
cal c
ondi
tions
. It i
s un
likel
y th
at a
ny o
bjec
tive
will
req
uire
con
ditio
n st
atem
ents
in a
ll fo
ur b
oxes
.5.
Lis
t onl
y cr
itica
l cri
teri
a.6.
Use
the
com
plet
ed w
orks
heet
to w
rite
you
r st
atem
ent o
f th
e in
stru
ctio
nal o
bjec
tive.
For
inte
rnal
use
, you
may
util
ize
the
wor
kshe
et to
com
mun
icat
e w
ith th
e ot
her
mem
bers
of
your
team
.
Rel
ated
Job
Aid
s:
1.E
ditin
g In
stru
ctio
nal O
bjec
tives
Use
this
che
cklis
t to
edit
and
revi
se y
our
stat
emen
t of
the
inst
ruct
iona
l obj
ectiv
e.2.
Tes
t Spe
cifi
catio
ns W
orks
heet
Use
the
obje
ctiv
e to
sel
ect t
he a
ppro
pria
te te
st f
orm
at.
107
106
44
y fi
t:
Inst
ruct
iona
l Obj
ectiv
es
Indi
vidu
al I
nstr
uctio
nal O
bjec
tives
Beh
avio
rC
ondi
tions
Cri
teri
a
TiP
..4
108
Set o
f In
stru
ctio
nal O
bjec
tives
Rel
evan
ce
+Su
ffic
ienc
y*A
ppro
pria
tene
ss
109
Edi
ting
Inst
ruct
iona
l Ob'
ectiv
es
Indi
vidu
al I
nstr
uctio
nal O
bjec
tives
I. B
ehav
ior.
Doe
s th
e ob
ject
ive
clea
rly
iden
tify
an o
bser
vabl
e. m
easu
rabl
e, r
epea
tabl
e be
havi
or w
hich
the
perf
orm
er w
illbe
abl
e to
dem
onst
rate
?
2. C
ondi
tions
. Doe
s th
e ob
ject
ive
spec
ify
the
impo
rtan
t cir
cum
stan
ces
in w
hich
the
perf
orm
ance
is to
take
plac
e, in
clud
ing
the
phys
ical
env
iron
men
t, eq
uipm
ent a
nd to
ols,
ref
eren
ces
and
job
aids
, and
inpu
ts f
rom
oth
ers
whi
ch a
re r
equi
red,
perm
itted
, or
deni
ed
to th
e pe
rfor
mer
?
3. C
rite
ria.
Doe
s th
e ob
ject
ive
set t
he s
tand
ard
for
the
min
imum
acc
epta
ble
perf
orm
ance
in te
rms
of s
uch
fact
ors
as ti
me
limit,
err
or
limit,
pro
duct
fea
ture
s, a
nd p
roce
dura
l req
uire
men
ts?
Set o
f In
stru
ctio
nal O
bjec
tives
4. R
elev
ance
. Are
all
the
enab
ling
obje
ctiv
es r
elev
ant t
o th
e ac
hiev
emen
: of
the
term
inal
obj
ectiv
e? A
re th
ete
rmin
al o
bjec
tives
rele
vant
to th
e ac
hiev
emen
t of
a co
rpor
ate
goal
?
5. S
uffi
cien
cy. D
o th
e en
ablin
g ob
ject
less
spe
cify
all
the
requ
ired
beh
avio
rs w
hich
add
up
to th
e pe
rfor
man
cere
quir
ed in
the
term
inal
obj
ectiv
e?
6. A
ppro
pria
tene
ss. A
re th
e la
ngua
ge, n
umbe
r an
d th
e le
vel o
f de
tail
of th
e se
t of
obje
ctiv
es s
uita
ble
for
the
inte
nded
aud
ienc
e an
d
Purp
ose?
145
Obj
ectiv
e T
ests
Gen
eral
1. I
s th
ere
a pl
ace
for
the
stud
ent's
nam
e an
d da
te?
2. I
s th
e co
urse
nam
e/nu
mbe
r an
d te
st n
ame
indi
cate
d?
3. A
re th
e pa
ges/
ques
tions
num
bere
d co
rrec
tly?
4. I
s th
e te
st a
ssem
bled
cor
rect
ly (
page
s in
ord
er)?
5. A
re th
ere
dire
ctio
ns o
n ho
w to
take
the
test
/sec
tions
of
the
test
?
6. H
as c
orre
ct g
ram
mar
bee
n us
ed in
all
test
item
s?
7. H
ave
vagu
e/am
bigu
ous
term
s be
en a
void
ed?
8. D
oes
each
test
item
con
tain
onl
y on
e id
ea?
9. I
s th
ere
only
one
cor
rect
ans
wer
for
eac
h te
st it
em?
10. I
s th
e te
st w
ritte
n at
the
trai
:iee'
s le
vel o
f co
mpr
ehen
sion
?
11. D
oes
the
corr
ect a
nsw
er to
one
que
stio
n gi
ve a
hin
t/cue
to th
e co
rrec
t ans
wer
of
anot
her?
12. A
re th
ere
dupl
icat
e qu
estio
ns?
13. A
re a
ll pa
rts
of th
e qu
estio
n on
the
sam
e pa
ge?
14. I
s a
mea
ns p
rovi
ded
to r
ecor
d th
e an
swer
to e
ach
item
(a
blan
k, a
n an
swer
she
et, e
tc.)
?
15. D
o th
e di
rect
ions
cle
arly
indi
cate
wha
t the
trai
nee
is to
do
with
the
diag
ram
/cha
rt?
16. I
f la
belin
g is
req
uire
d, a
re th
ere
spac
es p
rovi
ded?
113
112
46
Shor
t Ans
wer
/Fill
-In
Tes
t Ite
ms
1. I
s th
e ite
m c
onst
ruct
ed s
o th
at o
nly
one
brie
fly
wri
tten
answ
er is
pos
sibl
e?
2. F
or in
com
plet
e se
nten
ce it
ems,
doe
s en
ough
of
the
stat
emen
t rem
ain
to c
onve
y th
e in
tent
to th
e tr
aine
e?
3. D
oes
the
mai
n id
ea o
f th
e in
com
plet
e se
nten
ce p
rece
de th
e bl
ank
(bla
nk a
t/nea
r en
d of
sen
tenc
e)?
4. H
as o
nly
a si
gnif
ican
t wor
d/te
rm/s
ymbo
l bee
n om
itted
?
5. F
or n
umer
ical
ans
wer
s, h
as th
e de
gree
of
prec
isio
n re
quir
ed b
een
incl
uded
in th
e st
em?
6. H
ave
nega
tive
or d
oubl
e ne
gativ
e st
ems
been
avo
ided
?
7. H
as a
list
of
acce
ptab
le r
espo
nses
(va
riat
ions
of
answ
er th
at a
re c
onsi
dere
d co
rrec
t) b
een
incl
uded
in th
e an
swer
cod
e?
8. A
re th
ere
any
typo
grap
hica
l err
ors/
mis
spel
lings
?
Mul
tiple
-Cho
ice
Tes
t Ite
ms
1. H
ave
nega
tive
and
doub
le n
egat
ive
stem
s be
en a
void
ed?
2. H
ave
gram
mat
ical
clu
es b
een
avoi
ded?
3. I
s as
muc
h of
the
wor
ding
as
poss
ible
in th
e st
em r
athe
r th
an e
ach
dist
ract
er?
4. A
re a
ll di
stra
cter
s of
an
item
app
roxi
mat
ely
the
sam
e le
ngth
?
5. A
re a
ll di
stra
cter
s of
an
item
pla
usib
le?
6. A
re e
noug
h di
stra
cter
s pr
esen
t in
each
item
to r
educ
e gu
essi
ng?
7. A
re d
istr
acte
rs a
rran
ged
in a
n or
derl
y m
anne
r (a
lpha
num
eric
, chr
onol
ogic
al)?
8. H
ave
dist
ract
ers
such
as
"all
of th
e ab
ove,
non
e of
the
abov
e, A
and
B, e
tc."
bee
n el
imin
ated
?
9. H
ave
vagu
e w
ords
(m
ay, s
houl
d, c
ould
, mig
ht, u
sual
ly, n
orm
ally
, etc
.) b
een
avoi
ded?
10. A
re a
ll pa
rts
of th
e qu
estio
n/an
swer
on
the
sam
e pa
ge?
11
.A
re th
ere
typo
grap
hica
l err
ors/
mis
spel
lings
?
115
114
47
How
to S
elec
t the
Opt
imum
Del
iver
y St
rate
gy
1. D
efin
e th
e co
ntex
t in
whi
ch th
e co
urse
dev
elop
men
t pro
ject
is to
take
pla
ce. U
se d
ata
from
the
perf
orm
ance
, lea
rner
, and
task
anal
yses
con
duct
ed e
arlie
r to
ans
wer
thes
e qu
estio
ns:
Wha
t is
the
perf
orm
ance
pro
blem
for
whi
ch th
e co
urse
is to
pro
vide
a s
olut
ion?
Who
? W
ho a
re th
e ta
rget
per
form
ers/
stud
ents
for
the
prop
osed
cou
rse?
Who
will
dev
elop
the
cour
se?
Who
will
del
iver
the
cour
se?
Who
will
man
age
thes
e ac
tiviti
es?
Whe
re?
Whe
re a
re th
e ta
rget
per
form
ers
loca
ted?
Whe
re w
ill th
e co
urse
be
deliv
ered
?W
hen?
Whe
n is
the
impr
oved
per
form
ance
req
uire
d? W
hen
shou
ld th
e co
urse
be
on li
ne?
How
long
sho
uld
it la
st?
How
man
y tim
esw
ill it
be
offe
red?
How
fre
quen
tly w
ill it
be
offe
red?
Wha
t? W
hat a
re th
e re
sour
ces
and
cons
trai
nts
on th
e de
sign
, pro
duct
ion,
and
del
iver
y of
the
cour
se?
2. I
dent
ify
alte
rnat
ive
deliv
ery
stra
tegi
es. C
onsi
der
thes
e al
tern
ativ
es a
nd a
dd a
ny o
ther
s: S
ME
-led
trai
ning
; tel
etra
inin
g; m
ater
ials
-ba
sed,
inst
ruct
or-l
ed tr
aini
ng; s
elf-
pace
d pr
int;
self
-pac
ed n
onpr
int;
com
pute
r-ba
sed
trai
ning
; and
inte
ract
ive
vide
o.
3. S
elec
t fea
sibl
e al
tern
ativ
es. U
se th
e pr
evio
us d
ecis
ion
tabl
e an
d el
imin
ate
thos
e st
rate
gies
whi
ch a
re in
appr
opri
ate
for
the
prop
osed
cou
rse
4. E
stim
ate
effe
ctiv
enes
s of
eac
h al
tern
ativ
e de
liver
y st
rate
gy. U
se b
est a
vaila
ble
expe
rt e
stim
ates
of
the
effe
ctiv
enes
s. B
eco
nser
vativ
e ab
out t
he e
ffec
tiven
ess
of a
ny s
trat
egy
whi
ch y
ou p
refe
r. H
ere
are
som
e in
dica
tors
of
effe
ctiv
enes
s fo
r yo
u to
cons
ider
. You
may
wan
t to
add
othe
rs:
Impa
ct. E
cono
mic
impa
ct o
n th
e bo
ttom
line
of
the
orga
niza
tion.
Inc
reas
ed p
rodu
ctiv
ity. I
mpr
oved
job
perf
orm
ance
. Im
prov
ed jo
bsa
tisfa
ctio
n.O
utco
mes
. App
licat
ion
test
res
ults
. End
-of-
cour
se te
st r
esul
ts. U
nit t
est r
esul
ts.
Proc
ess.
Stu
dent
per
form
ance
dur
ing
cour
se. S
tude
nt s
atis
fact
ion.
Ins
truc
tor
perf
orm
ance
. Ins
truc
tor
satis
fact
ion.
5. E
stim
ate
cost
s of
eac
h al
tern
ativ
e de
liver
y st
rate
gy. U
se th
e be
st a
vaila
ble
estim
ates
for
the
base
cos
ts. U
se th
e ne
xt w
orks
heet
for
com
putin
g to
tal c
ost f
igur
es.
117
11E
;48
6. C
ompa
re th
e al
tern
ativ
e de
liver
y st
rate
gies
on
cost
s an
d ef
fect
iven
ess.
Se
lee-
, the
del
iver
y st
rate
gy w
hich
can
pro
vide
mos
tef
fect
iven
ess
at le
ast c
ost
119
118
49
How
To
Sele
ct th
e O
ptim
um D
eliv
ery
Stra
tegy
Use
rs.
Tea
cher
s/In
stru
ctor
s
Purp
ose:
To
sele
ct th
e op
timum
str
ateg
y to
del
iver
a p
ropo
sed
cour
se.
Inpu
ts:
Lea
rner
and
task
ana
lysi
s da
ta.
Skill
/kno
wle
dge
requ
irem
ents
wor
kshe
ets.
Info
rmat
ion
on f
easi
ble
deliv
ery
stra
tegi
es.
Dir
ectio
ns:
Use
this
che
cklis
t in
conj
unct
ion
with
the
prec
edin
g de
cisi
on ta
ble
and
the
follo
win
g m
atri
xR
ead
thro
ugh
the
chec
klis
t to
beco
me
fam
iliar
with
the
proc
edur
e.M
odif
y th
e ch
eckl
ist t
o su
it yo
ur lo
cal n
eeds
.R
efer
to th
e ch
eckl
ist a
s yo
u w
ork
thro
ugh
the
proc
edur
e. C
heck
off
eac
h st
ep a
s yo
u co
mpl
ete
it.
Rel
ated
Job
Aid
s:T
rain
ing
Del
iver
y St
rate
gies
121
50
USE
this
del
iver
y st
rate
gySM
E-L
ED
TR
AIN
ING
TE
LE
TR
AIN
ING
Tra
inin
g D
eliv
ery
Stra
tegi
es
IF th
ese
cond
ition
s ar
e pr
esen
tSe
vera
l SM
Es
are
avai
labl
e.O
nly
a lim
ited
num
ber
of s
tude
nts
need
to b
e tr
aine
d.T
rain
ing
is n
eede
d ur
gent
ly.
Bud
get i
s lim
ited.
Stud
ents
are
loca
ted
at th
e tr
aini
ng s
ite.
Com
pete
nt in
stru
ctio
nal d
esig
ners
are
not
ava
ilabl
e.C
ours
e co
nten
t i. l
ikel
y to
cha
nge
in th
e ne
ar f
utur
e.R
equi
res
appr
oxim
atel
y 20
- 4
0 ho
urs
of d
evel
opm
ent f
or o
neho
ur o
f de
liver
y.
Seve
ral s
tude
nts
need
to b
e tr
aine
d.St
uden
ts a
re lo
cate
d in
dif
fere
nt r
emot
e ar
eas.
Onl
y a
few
SM
Es
are
avai
labl
e.H
ardw
are
and
tech
nica
l per
sonn
el a
re a
vaila
ble.
Loc
al lo
gist
ic a
nd in
stru
ctio
nal s
uppo
rt a
re a
vaila
ble.
Cou
rse
cont
ent i
s lik
ely
to c
hang
e in
the
near
fut
ure.
Req
uire
s ap
prox
imat
ely
20 -
80
hour
s of
dev
elop
men
t for
one
hour
of
deliv
ery.
MA
TE
RIA
LS-
BA
SED
,C
onte
nt is
not
like
ly to
cha
nge
in th
e ne
ar f
utur
e.IN
STR
UC
TO
R-L
ED
In.tr
ucto
rs c
an b
e re
crui
ted
and
trai
ned
easi
ly a
nd a
re w
illin
g to
fol
low
dir
ectio
nsT
RA
ININ
GE
noug
h tim
e an
d m
oney
is a
vaila
ble
for
the
prod
uctio
n of
inst
ruct
iona
l mat
eria
ls.
Com
pete
nt c
ours
e de
velo
pers
and
team
mem
bers
are
ava
ilabl
e.R
equi
res
appr
oxim
atel
y 40
- 8
0 ho
urs
of d
evel
opm
ent f
or o
neho
ur o
f de
liver
y.
123
122
51
SEL
F-PA
CE
D P
RIN
TC
onte
nt is
not
like
ly to
cha
nge
in th
e ne
ar f
utur
e.la
rge
num
bers
of
stud
ents
hav
e to
be
trai
ned.
Stud
ents
are
loca
ted
in d
iffe
rent
are
as.
Stud
ents
hav
e su
ffic
ient
ly h
igh
read
ing
abili
ties.
Bud
get f
or m
ater
ials
dev
elop
men
t is
limite
d.C
ompe
tent
inst
ruct
iona
l des
igne
rs a
nd w
rite
rs a
re a
vaila
ble.
Req
uire
s ap
prox
imat
ely
60 -
80
hour
s of
dev
elop
men
t for
one
hour
of
deliv
ery.
SEL
F-PA
CE
D N
ON
PRIN
TC
onte
nt is
not
like
ly to
cha
nge
in th
e ne
ar f
utur
e.St
uden
ts a
re lo
cate
d in
dif
fere
nt a
reas
.St
uden
ts h
ave
low
rea
ding
abi
litie
s.A
dequ
ate
budg
et f
or m
ater
ials
dev
elop
men
t is
avai
labl
e.C
ompe
tent
inst
ruct
iona
l des
igne
rs, w
rite
rs a
nd m
edia
pro
duct
ion
peop
le a
reav
aila
ble.
Med
ia e
quip
men
t nee
ds is
ava
ilabl
e at
dif
fere
nt lo
catio
ns.
Req
uire
s ap
prox
imat
ely
80-1
00 h
ours
of
deve
lopm
ent f
or o
neho
ur o
f de
liver
y.
CO
MPU
TE
R-B
ASE
D T
RA
ININ
GL
arge
and
or
geog
raph
ical
ly d
ispe
rsed
pop
ulat
ion.
Dif
ficu
lt fo
r st
uden
ts to
leav
e jo
b fo
r tr
aini
ng, o
r tim
ely
deliv
ery
is c
ritic
alJo
b/T
asks
are
cri
tical
req
uiri
ng m
aste
ry b
efor
e pe
rfor
man
ce in
a li
ve e
nvir
onm
ent
Cri
tical
asp
ects
of
perf
orm
ance
can
be
sim
ulat
ed u
sing
com
pute
r te
xt, g
raph
ics,
anim
atio
nA
bilit
y to
mak
e an
d di
ssem
inat
e m
inor
con
tent
cha
nges
qui
ckly
is d
esir
able
.C
ompe
tent
CB
T s
peci
alis
ts a
nd o
ther
per
sonn
el a
re a
vaila
ble
Req
uire
s ap
prox
. 100
-150
hrs
. of
deve
lopm
ent f
or e
ach
hour
of
deliv
ery.
124
125
52
INT
ER
AC
TIV
E V
IDE
O
126
Lar
ge a
nd/o
r ge
ogra
phic
ally
dis
pers
ed ta
rget
pop
ulat
ion
Dif
ficu
lt fo
r st
uden
ts to
leav
e jo
b fo
r tr
aini
ng, o
r tim
ely
deliv
ery
is c
ritic
alJo
b/T
asks
are
cri
tical
req
uiri
ng m
aste
ry b
efor
e pe
rfor
man
ce in
a li
ve e
nvir
onm
ent
Sim
ulat
ions
of
criti
cal j
ob ta
sks
and/
or in
terp
erso
nal s
kill
requ
ire
the
real
ism
of
vide
o/au
dio
Ade
quat
e bu
dget
and
dev
elop
men
t tim
e is
ava
ilabl
eA
bilit
y to
mak
e an
d di
ssem
inat
e m
inor
con
tent
cha
nges
qui
ckly
is d
esir
able
.C
ompe
tent
inte
ract
ive
vide
o sp
ecia
lists
and
oth
er p
erso
nnel
are
ava
ilabl
eR
equi
res
appr
ox. 1
50 2
50 h
rs. o
f de
velo
pmen
t for
eac
h ho
ur o
f de
liver
y.
127
53
OP'
m/0
ME
DIA
SE
LE
CT
ION
*Ide
ntif
y m
edia
attr
ibut
es r
equi
red
by th
ein
stru
ctio
nal o
bjec
tives
*Ide
ntif
y m
edia
attr
ibut
es r
equi
red
by s
tude
nts
* D
eter
min
e m
edia
res
ourc
es a
nd c
onst
rain
tsde
liver
y sy
stem
; av
aila
ble
to y
our
stud
ents
prod
uctio
n re
sour
ces
cost
s, a
nd c
onst
rain
ts in
you
ror
gani
zatio
n
*Mak
ea
list o
f th
e re
quir
ed m
edia
attr
ibut
es,
reso
urce
s,an
d co
nstr
aint
s*S
elec
t the
app
ropr
iate
med
ium
or
med
iaco
mbi
natio
n ba
sed
on th
is li
st.
128
129
ME
DIA
SE
LE
CT
ION
I. I
dent
ify
med
ia a
ttrib
utes
req
uire
d by
the
inst
ruct
iona
l obj
ectiv
es. F
or e
xam
ple,
the
task
may
req
uire
col
or a
nd s
ound
2. I
dent
ify
med
ia a
ttrib
utes
req
uire
d by
stu
dent
s. U
se th
e in
form
atio
n co
llect
ed d
urin
g le
arne
r an
alys
is. F
or e
xam
ple,
if y
our
stud
ents
do
not h
ave
high
rea
ding
abi
lity,
the
med
ia a
ttrib
ute
of tr
ansm
ittin
g te
xt is
irre
leva
nt.
3. D
eter
min
e m
edia
res
ourc
es a
nd c
onst
rain
ts in
the
deliv
ery
syst
em. F
or e
xam
ple,
if th
e tr
aini
ng c
ente
rs h
ave
vide
o eq
uipm
ent,
this
sho
uld
be ta
ken
into
acc
ount
.
4. D
eter
min
e m
edia
res
ourc
es a
vaila
ble
to y
our
stud
ents
. For
exa
mpl
e, m
ost s
tude
nts
have
acc
ess
to a
udio
cass
ette
pla
yers
at
hom
e or
in th
eir
auto
mob
iles.
5. D
eter
min
e m
edia
pro
duct
ion
reso
urce
s co
sts,
and
con
stra
ints
in y
our
orga
niza
tion.
For
:-.x
.am
ple,
if y
ou d
o no
t hav
e an
yte
levi
sion
cam
eras
, you
may
not
wan
t to
unde
rtak
e lo
cal v
ideo
tape
pro
duct
ion.
6. D
eter
min
e m
edia
pro
duct
ion
tale
nts
and
skill
s am
ong
the
mem
bers
of
your
team
.
7. M
ake
a lis
t of
the
requ
ired
med
ia a
ttrib
utes
, res
ourc
es, a
nd c
onst
rain
ts. S
eInt
the
appr
opri
ate
med
ium
or
med
ia c
ombi
natio
nba
sed
on th
is li
st.
131
130
54
Sele
ctin
g In
stru
ctio
nal M
etho
ds
IF th
ese
cond
ition
s ar
e pr
esen
t.
.
Del
iver
y sy
stem
is g
roup
-pac
ed.
Tra
ined
inst
ruct
or a
vaila
ble.
Inst
ruct
or G
uide
ava
ilabl
e.
Del
iver
y sy
stem
is g
roup
-pac
ed.
Stud
ents
hav
e co
nsid
erab
le e
xper
ienc
e an
d ex
pert
ise.
Inst
ruct
or c
apab
le o
f fa
cilit
atin
g gr
oup
activ
ities
.
TH
EN
con
side
r us
ing
thes
e in
stru
ctio
nal m
etho
ds.
Del
iver
y sy
stem
is g
roup
- p
aced
.T
rans
fer
of tr
aini
ng im
port
ant.
Stud
ents
hav
e co
nsid
erab
le e
xper
ienc
e an
d ex
pert
ise.
Inst
ruct
or c
apab
le o
f fa
cilit
atin
g gr
oup
activ
ities
.
Del
iver
y sy
stem
cou
ld b
e gr
oup-
or
indi
vidu
al-p
aced
.SM
E m
aste
r pe
rfor
mer
is a
vaila
ble.
Con
tent
invo
lves
use
of
equi
pmen
t and
tool
s
Del
iver
y sy
stem
is s
elf
- pa
ced.
Inst
ruct
ors/
SME
s no
t ava
ilabl
e.St
uden
ts d
ispe
rsed
ove
r a
wid
e ar
ea.
132
Lec
ture
Dis
cove
ry L
earn
ing
Dis
cuss
ion
Inqu
iry
Met
hod
Deb
rief
ing
Inst
ruct
iona
l Gam
ePe
er T
utor
ing
Rol
e-pl
aySt
ruct
ured
or
Gui
ded
Dis
cuss
ion
Cas
e st
udy
Cri
tical
Inc
iden
t Pro
cess
Sim
ulat
ion
Lab
orat
ory
Exp
erie
nce
Mod
elin
gO
n-th
e Jo
b T
rain
ing
Supe
rvis
ed P
ract
ice
Adj
unct
stu
dy g
uide
Dri
ll an
d Pr
actic
e
Inde
pend
ent S
tudy
Lea
rner
-con
trol
led
Inst
ruct
ion
Prog
ram
med
Ins
truc
tion
Adj
unct
Stu
dy G
uide
s
Cas
e St
udy
Cri
tical
Inc
iden
t Pro
cess
Deb
rief
ing
Dem
onst
ratio
n
134
Inst
ruct
iona
l Met
hods
Inst
ruct
iona
l sup
plem
ents
whi
ch e
nabl
e us
to c
onve
rt to
exi
stin
g m
ater
ials
for
inde
pend
ent s
tudy
. Thi
s m
etho
d pr
ovid
es in
stru
ctio
nal o
bjec
tives
, cri
teri
onqu
estio
ns, p
ract
ice
exer
cise
s, s
tudy
sug
gest
ions
, tra
nsfe
r as
sign
men
ts, a
ndst
ruct
ured
not
e-ta
king
mat
eria
ls to
sup
port
one
or
mor
e te
xtbo
oks
and
othe
rin
stru
ctio
nal m
ater
ials
.
In th
is m
etho
d, th
e st
uden
t is
pres
ente
d a
wel
l-do
cum
ente
d de
scri
ptio
n of
asi
mul
ated
or
real
-lif
e si
tuat
ion.
The
lear
ning
exp
erie
nce
com
es th
roug
hre
cogn
ition
, ana
lysi
s, a
nd s
olut
ion
of th
e pr
oble
m w
ritte
n in
to th
e ca
se. T
heca
ses
may
be
wor
ked
indi
vidu
ally
; the
y m
ay a
lso
be c
ompl
eted
in g
roup
s to
allo
win
tera
ctio
n an
d fe
edba
ck o
f pa
rtic
ipan
ts.
The
inci
dent
pro
cess
is a
var
iatio
n of
the
case
stu
dy a
nd d
iffe
rs f
rom
it p
rim
arily
inth
e am
ount
of
info
rmat
ion
give
n th
e pa
rtic
ipan
t. In
the
case
stu
dy, a
lldo
cum
enta
tion
is p
rovi
ded;
in th
e in
cide
nt p
roce
ss, t
he p
artic
ipan
t is
prov
ided
only
an
inci
dent
cri
tical
in th
e de
velo
pmen
t of
the
prob
lem
and
mus
t see
k ou
t the
addi
tiona
l fac
ts r
equi
red
(usu
ally
fro
m th
e in
stru
ctor
).
An
inst
ruct
iona
l act
ivity
whi
ch f
ollo
ws
any
othe
r in
tens
e, in
tera
ctiv
e in
stru
ctio
n(s
uch
as r
ole-
play
ing)
in w
hich
the
inst
ruct
or a
sks
a se
ries
of
ques
tions
to e
licit
impo
rtan
t ins
ight
s fr
om s
tude
nts.
Deb
rief
ing
ques
tions
usu
ally
dra
w a
ttent
ion
toth
e da
ta f
rom
the
stud
ents
' exp
erie
nces
in th
e ea
rlie
r ac
tivity
so
that
they
can
disc
over
rel
evan
t pri
ncip
les.
A m
aste
r pe
rfor
mer
sho
ws
how
to u
se a
n eq
uipm
ent o
r to
ol, e
xpla
inin
g ea
ch s
tep
of th
e pr
oced
ure.
He
or s
he a
lso
expl
ains
the
actio
ns, d
ecis
ions
, and
pre
caut
ions
requ
ired
so
that
stu
dent
s ca
n fo
llow
the
proc
edur
e an
d un
ders
tand
the
ratio
nale
for
vari
ous
step
s.
135
56
Dis
cove
ry L
earn
ing
Dis
cuss
ion
Dri
ll an
d Pr
actic
e
Inde
pend
ent S
tudy
Inqu
iry
Met
hod
Inst
ruct
iona
l Gam
e
Lab
orat
ory
Exp
erie
nce.
Lea
rner
-Con
trol
led
Inst
ruct
ion
Thi
s m
etho
d us
es a
ser
ies
of a
ctiv
ities
or
ques
tions
to le
ad th
e st
uden
t to
disc
over
basi
c co
ncep
ts, p
rinc
iple
s, a
nd r
ules
in a
n in
stru
ctio
nal a
rea.
The
lead
ing
expe
rien
ce in
the
unst
ruct
ured
dis
cuss
ion
is c
ontr
olle
d by
the
part
icip
ants
who
als
opr
ovid
e th
e su
bjec
t mat
ter
expe
rtis
e. P
artic
ipan
ts a
lso
prov
ide
the
dire
ctio
n an
d ef
fort
tow
ards
reac
hing
the
desi
red
goal
.
Rep
eate
d pr
esen
tatio
n of
mat
eria
l and
test
of
acqu
isiti
on u
ntil
resp
onse
s m
eet c
rite
ria.
Rea
ding
, ins
truc
tion,
pra
ctic
e, e
tc. a
re d
one
at th
e di
scre
tion
of th
e st
uden
t with
out g
uida
nce
from
the
inst
ruct
or
Pres
enta
tion
of d
ata,
info
rmat
ion,
etc
. in
resp
onse
to s
tude
nt in
quir
es I
n so
me
case
s, th
e fo
rm a
nd o
rder
of
pres
enta
tion
is a
dapt
ive
base
d on
pre
viou
sre
spon
ses
by th
e st
uden
t.
A s
mal
l-gr
oup
inst
ruct
iona
l tec
hniq
ue w
hich
is c
hara
cter
ized
by
rule
s fo
r pl
ayin
gan
d fo
r w
inni
ng. G
ame:
, usu
ally
inco
rpor
ate
an e
lem
ent o
f co
nflic
t (ag
ains
t oth
erpl
ayer
s or
aga
inst
tim
e lim
its, f
or e
xam
ple)
. Win
ning
the
gam
e is
rel
ated
to th
em
aste
ry o
f th
e in
stru
ctio
nal o
bjec
tives
. If
the
gam
e re
flec
ts s
ome
real
-wor
ldpr
oces
s, it
is c
alle
d an
inst
ruct
iona
l sim
ulat
ion
gam
e.
Part
icip
ants
pra
ctic
e ut
ilizi
ng th
e ac
tual
or
sim
ulat
ed v
ersi
ons
of e
quip
men
t,m
ater
ials
, and
pro
cedu
res
that
they
will
ulti
mat
ely
use
on th
e jo
b.
Thi
s m
etho
d al
low
s st
uden
ts to
mak
e ke
y le
arni
ng d
ecis
ion
on p
ace,
se
"uen
ce,
cont
ent,
and
met
hod
of e
valu
atio
n. L
earn
er-c
ontr
olle
d in
stru
ctio
n in
corp
orat
es a
vari
ety
of in
stru
ctio
nal m
ater
ials
in p
rovi
ding
opt
ions
to s
tude
nts.
Lec
ture
Ora
l pre
sent
atio
n of
inst
ruct
iona
l con
tent
to s
tude
nts
who
can
onl
y re
spon
d
136
137
57
Mod
elin
g
On-
the
Job
Tra
inin
g
inte
rmitt
ently
dur
ing
that
pre
sent
atio
n.
The
stu
dent
is s
how
n ho
w s
omet
hing
ope
rate
s, is
rep
aire
d, e
tc. b
y be
ing
show
nan
d le
d th
roug
h th
e ap
prop
riat
e st
eps,
act
ions
, and
dec
isio
ns r
equi
red.
Mod
elin
g pl
aces
em
phas
is o
n st
uden
t per
form
ance
(im
itatio
n) f
ollo
win
g an
exp
ert's
perf
orm
ance
(m
odel
).
On-
the
job
trai
ning
is th
e be
st p
ossi
ble
way
to b
ring
a tr
aine
e in
to c
lose
con
tact
with
the
job
whi
le u
nder
trai
ning
. The
sup
ervi
sor
or a
n ex
peri
ence
d w
orke
r se
rves
as th
e in
stru
ctor
. The
trai
nee
is s
how
n ho
w jo
b pr
oced
ures
are
car
ried
out
. The
stud
ent i
s gr
adua
lly m
ade
to d
o m
ore
and
mor
e of
the
job.
For
a li
mite
d nu
mbe
rof
jobs
, or
lear
ners
, thi
s ty
pe o
f tr
aini
ng is
fea
sibl
e an
d ad
equa
te.
Peer
Tut
orin
gA
n in
stru
ctio
nal s
trat
egy
in w
hich
stu
dent
s ar
e se
lect
ed a
nd tr
aine
d to
teac
h le
ss a
dvan
ced
stud
ents
or th
eir
peer
s. T
utor
ing
has
been
sho
wn
to b
enef
it bo
th th
e tu
tor
and
his
or h
er s
tude
nt.
Prog
ram
med
Ins
truc
tion
Prim
arily
des
igne
d fo
r in
divi
dual
ized
, sel
f-pa
ced
.trai
ning
. Mat
eria
l is
prep
ared
and
pre
sent
ed in
ase
ries
of
wel
l-pl
anne
d se
quen
tial s
teps
. The
trai
nee
mus
t cho
ose
a co
rrec
t sta
tem
ent f
rom
a g
iven
set
of s
tate
men
ts, f
ill in
bla
nks,
wri
te o
ut a
nsw
ers
to q
uest
ions
, or
perf
orm
in a
ny w
ay r
equi
red.
With
this
trai
ning
met
hod,
the
stud
ent g
ets
imm
edia
te f
eedb
ack.
Upo
n gi
ving
the
corr
ect r
espo
nse,
he
orsh
e re
ceiv
es in
form
atio
n as
to it
s co
rrec
tnes
s. T
he tr
aine
e ca
n th
en p
roce
ed to
the
next
ste
p. I
n th
eev
ent t
hat t
he w
rong
res
pons
e is
giv
en, t
he tr
aine
e is
told
to r
evie
w th
e m
ater
ial a
nd to
giv
e an
othe
ran
swer
, or
he is
giv
en a
n ex
plan
atio
n of
why
the
resp
onse
was
wro
ng.
Rol
e-pl
ay
13b
The
lear
ning
exp
erie
nce
in r
ole-
play
ing
is p
rim
arily
inte
ract
ive.
A p
artic
ular
situ
atio
n is
pre
sent
ed, a
nd p
artic
ipan
ts a
ssum
e sp
ecif
ic c
hara
cter
rol
es in
the
inte
ract
ion.
Rol
e-pl
ayin
g ty
pica
lly a
ttem
pts
to s
imul
ate
the
job
envi
ronm
ent.
139
58
Sim
ulat
ion
Stru
ctur
ed o
rG
uide
d D
iscu
ssio
n
Part
icip
ants
, ind
ivid
ually
or
as te
ams,
rep
rese
nt d
ecis
ion
mak
ers
in a
nor
gani
zatio
n. T
hey
mak
e th
e sa
me
kind
of
deci
sion
s th
ey m
ight
mak
e in
a r
eal
situ
atio
n bu
t the
dat
a an
d th
e en
viro
nmen
t are
sim
ulat
ed. S
imul
atio
n is
oft
endo
ne b
y m
eans
of
gam
es w
hich
can
be
pape
r-an
d-pe
ncil
exer
cise
s or
com
pute
rize
d ac
tiviti
es. T
hese
sim
ulat
ion
gam
es c
lose
ly a
ppro
xim
ate
the
real
wor
ld in
that
the
data
use
d by
the
trai
nees
is th
e sa
me,
but
they
nor
mal
ly o
pera
tein
a c
ompr
esse
d tim
e fr
ame.
Que
stio
ns p
repa
red
in a
dvan
ce a
re d
irec
ted
to s
tude
nts
to g
uide
the
disc
ussi
onal
ong
a pr
econ
ceiv
ed p
ath.
The
dis
cuss
ion
is r
estr
icte
d w
ithin
pre
dete
rmin
edbo
unda
ries
to e
nsur
e th
e gr
oup
stay
s on
trac
k.
Sugg
estiv
e-A
ccel
erat
ive
Initi
ally
use
d fo
r la
ngua
ge tr
aini
ng, t
his
met
hod
com
bine
s re
laxa
tion,
pos
itive
Lea
rnin
gsu
gges
tions
, mus
ic b
ackg
roun
ds, h
igh
mul
tisen
sory
inpu
ts, a
nd v
isua
l im
ager
y to
incr
ease
con
cent
ratio
n an
d to
red
uce
stre
ss a
nd a
nxie
ty.
Supe
rvis
ed P
ract
ice
Prac
tice
is d
one
unde
r di
rect
sup
ervi
sion
of
som
eone
suc
h as
an
inst
ruct
or o
rsu
perv
isor
.
141
1-10
59
Des
ign
Spec
ific
atio
ns R
epor
t
I. I
ntro
duct
ion
1. P
roje
ct b
ackg
roun
d (i
nclu
ding
sig
nifi
cant
con
stra
ints
)
2. C
ours
e ou
tline
3. C
ours
e sc
hedu
le
4. S
umm
ary
of p
erfo
rman
ce a
naly
sis
data
5. S
umm
ary
of le
arne
r an
alys
is d
ata
6. P
ropo
sed
deliv
ery
syst
em
7. N
otes
on
inst
ruct
iona
l med
ia a
nd m
etho
ds
8. T
est s
peci
fica
tions
(or
test
s)
9. P
reco
urse
mat
eria
ls a
nd a
ctiv
ities
II. D
esig
n Sp
ecif
icat
ions
for
Eac
h L
esso
n
1. T
itle
2. T
ime
3. P
rere
quis
ites
4. O
bjec
tives
5 C
onte
nt o
utlin
e6
Inst
ruct
iona
l act
iviti
es (
even
ts o
f in
stru
ctio
n)
7 In
stru
ctio
nal m
ater
ials
:St
uden
t Mat
eria
ls (
Stud
ent G
uide
, wor
kboo
k, f
lopp
y di
sk, e
tc.)
Inst
ruct
or M
ater
ials
(In
stru
ctor
Gui
de, A
dmin
istr
ator
Gui
de, e
tc.)
Med
ia (
tran
spar
enci
es, v
ideo
tape
s, s
lides
, etc
.)
142
143
60
Eve
ntIn
tent
ion
Eve
nts
of I
nstr
uctio
n
Cog
nitiv
e Ps
ycho
logi
cal P
roce
ss
Mot
ivat
ion:
Ori
enta
tion:
Prer
equi
site
s:
Pres
enta
tion:
Lea
rnin
gG
uida
nce:
Prac
tice:
Feed
back
:
Ass
essm
ent:
Ret
entio
n &
Tra
nsfe
r:
Get
the
stud
ents
rea
dy to
lear
n.
Prev
iew
the
cour
se, p
rovi
ding
an
over
view
of
the
expe
ctat
ions
for
he s
tude
nts
and
revi
ew th
e in
stru
ctio
nal o
bjec
tives
.
Rec
all p
rior
inst
ruct
ion
or p
rovi
de s
uppo
rtin
g sk
ill o
r kn
owle
dge
that
will
hel
p th
e st
uden
ts le
arn
the
new
mat
eria
l.
Pres
ent t
he n
ew m
ater
ial.
Prov
ide
inst
ruct
iona
l hel
p to
ass
ist s
tude
nts
in a
chie
ving
mas
tery
of
the
mat
eria
l.
Req
uire
the
stud
ents
to p
ract
ice
inte
grat
ing
and
perf
ectin
g ne
wsk
ills
and
know
ledg
e.
Prov
ide
accu
rate
and
tim
ely
info
rmat
ion
to th
e st
uden
ts a
bout
thei
r pe
rfor
man
ce.
Eva
luat
e th
e st
uden
ts p
erfo
rman
ce
Prep
air
patte
rns
of n
eura
l im
puls
es
Act
ivat
e a
proc
ess
of e
xect
uive
con
trol
Ret
revi
al o
f pr
ior
lear
ning
to w
orki
ng m
emor
y
Em
phai
ze f
eatu
res
for
sele
ctiv
e pe
rcep
tion
Sem
antic
Enc
odin
g &
cue
s fo
r re
trev
ial
Act
ivat
e re
spon
se g
ener
ator
s
Est
ablis
h re
info
rcem
ent
Act
ivat
e re
trev
ial s
ituat
iona
l rei
nfor
cem
ent
Giv
e op
port
uniti
es to
the
stud
ents
to in
tegr
ate
wha
t the
y ha
veIn
tegr
ate
with
pri
or k
now
ledg
e &
pro
vide
cue
sle
arne
d in
to th
eir
jobs
.
144
and
stra
tegi
es f
or r
etre
vial
145
61
Lea
rnin
gA
ctiv
ities
HA
ND
S O
FF
!!!
Tra
inin
g D
ept.
Pro
pert
y!!
146
BE
ST
CO
PY
AV
AIL
AB
LE14
7
Lan
guag
e R
evie
wpo
orex
celle
nt1.
Org
aniz
atio
n of
con
tent
. (A
re th
e le
sson
s an
d ot
her
units
of
12
34
5
inst
ruct
ion
logi
cally
org
aniz
ed?)
2. U
se o
f su
bhea
ding
s. (
Doe
s th
e in
stru
ctio
nal m
ater
ials
use
a1
23
45
cons
iste
nt s
et o
f he
adin
gs to
rev
eal t
he s
truc
ture
to th
e st
uden
ts?)
3 L
engt
h of
less
ons.
(A
re th
e le
sson
of
a su
itabl
e le
ngth
to p
erm
it1
23
45
mas
tery
in a
sui
tabl
e pe
riod
of
time?
)
4. C
lari
ty o
f pr
esen
tatio
n. (
Doe
s th
e co
urse
pre
sent
info
rmat
ion
in a
12
34
5
plai
n la
ngua
ge a
nd a
void
unn
eces
sary
jarg
on?)
5 C
onci
se s
tyle
. (D
oes
the
cour
se p
rese
nt in
form
atio
n in
a le
an a
nd1
23
45
succ
inct
fas
hion
?)
6. P
arag
raph
ing.
(A
re th
e re
adin
g m
ater
ials
div
ided
into
sho
rt, l
ogic
al1
23
45
para
grap
hs?)
7. U
se o
ften
tech
nica
l ter
ms.
(A
re te
chni
cal t
erm
s us
ed e
ffic
ient
ly?
Is e
ach
I2
34
5
term
cle
arly
def
ined
dur
ing
its f
irst
usa
ge?)
8. C
onsi
sten
cy. (
Are
dif
fere
nt te
chni
cal t
erm
s us
ed in
a c
onsi
sten
t1
23
45
fash
ion?
Are
the
less
ons
orga
nize
d in
a c
onsi
sten
t for
mat
?)
9. E
vide
nce
of p
roof
read
ing.
(A
re th
e co
urse
mat
eria
ls f
ree
from
12
34
5
lang
uage
and
typo
grap
hica
l err
ors?
)
10. U
se o
f ill
ustr
atio
ns. (
Are
illu
stra
tions
use
d fo
r cl
arif
icat
ion
rath
er1
23
45
than
for
dec
orat
ion
?)
149
4862
Tes
t Spe
cifi
catio
ns
Cou
rse
Titl
e:
Tes
t Titl
e (o
r N
umbe
r):
Skill
s/kn
owle
dge
cove
red
by th
e te
st:
Inst
ruct
iona
l obj
ectiv
es c
over
ed b
y th
e te
st:
Inst
ruct
iona
l con
tent
cov
ered
by
the
test
:
Tim
e re
quir
emen
t:
Tes
t pla
cem
ent i
n re
latio
nshi
p to
trai
ning
:
Typ
e of
test
/test
item
s:
Tes
t adm
inis
trat
ion
note
s:
Follo
w-u
p no
tes:
150
151
63
Inst
ruct
iona
l Des
ign
Eva
luat
ion
M:,;
'9'0
W '
Com
preh
ensi
ve &
val
id te
sts
and
exe
*Con
firm
ed D
eliv
ery
Stra
tegy
Med
ia A
ppro
pria
tnes
sE
vent
s of
Ins
truc
tion
-'.."
--.'
152
BE
ST
CO
PY
AV
AIL
AB
LE15
3
Inst
ruct
iona
l Des
ign
Shor
tcut
s
1.A
bbre
viat
e in
stru
ctio
nal d
esig
n sp
ecif
icat
ions
.
2.D
o sp
ecif
icat
ions
for
larg
er u
nits
of
inst
ruct
ion.
3.U
se s
ubje
ct-m
atte
r ex
pert
s as
inst
ruct
ors
or a
dmin
istr
ator
s.
4.R
equi
re S
ME
s to
del
iver
the
cour
se f
rom
und
etai
led
Inst
ruct
or G
uide
.
5Pr
oduc
e ea
sily
dev
elop
ed m
ater
ials
(e.
g., i
nstr
ucto
r-le
d in
stea
d of
sel
f-pa
ced;
pri
nt in
stea
d of
ela
bora
te m
edia
).
6B
orro
w f
rom
exi
stin
g co
urse
s w
hene
ver
poss
ible
.
7In
corp
orat
e do
cum
enta
tion
and
job
aids
in th
e co
urse
mat
eria
ls o
r us
e th
em in
stea
d of
the
cour
se m
ater
ials
.
155
154
64
MA
TE
RIA
L D
EV
EL
OPM
EN
T
App
eara
nce
and
effe
ct o
f al
l ins
truc
tiona
l mat
eria
ls is
the
ultim
ate
resp
onsi
bilit
y of
the
Des
igne
r/T
each
er. T
his
incl
udes
the
stud
ent a
nd in
stru
ctio
nal g
uide
. It i
s hi
s re
spon
sibi
lity
for
all m
ater
ials
and
his
nam
e sh
ould
app
ear
on a
ll m
ater
ials
. As
an a
id to
revi
ew th
e fi
nal p
rodu
ct a
nd b
efor
e im
plem
enta
tion
the
follo
win
g sh
ould
be
answ
ered
:
1. I
s th
e de
liver
y st
rate
gy th
e m
ost c
ost e
ffec
tive
one?
2 A
re th
e co
urse
obj
ectiv
es u
nder
stan
dabl
e an
d st
ated
in a
n ac
cept
able
for
mat
?
3 D
o th
e te
sts
dire
ctly
mea
sure
the
achi
evem
ent o
f th
e ob
ject
ives
?
4. D
oes
the
cour
se p
rese
nt a
ll cr
itica
l inf
orm
atio
n re
late
d to
the
obje
ctiv
es r
athe
r th
an n
ice
to k
now
, or
triv
ia?
5. D
oes
the
cour
se u
se th
e m
ost c
ost e
ffec
tive
com
bina
tion
of m
edia
? an
d m
etho
ds?
6. D
oes
the
cour
se in
clud
e ap
prop
riat
e le
arni
ng a
ctiv
ities
for
the
diff
eren
t eve
nts
of in
stru
ctio
n?
7. D
o th
e m
edia
ach
ieve
hig
h pr
oduc
tion
stan
dard
s?
8. I
s th
e di
ffic
ulty
leve
l of
the
cour
se a
nd m
ater
ial s
uite
d fo
r th
e ta
rget
pop
ulat
ion?
9. D
oes
the
cour
se p
rovi
de o
ppor
tuni
ties
for
the
prac
tice
and
revi
ew o
f cr
itica
l ski
lls a
nd k
now
ledg
e?
10. D
oes
the
cour
se h
ave
prov
isio
ns f
or ti
mel
y fe
edba
ck to
the
stud
ents
on
his
perf
orm
ance
?
1 1.
Doe
s th
e co
urse
em
phas
ize
the
appl
icat
ion
of s
kill
and
know
ledg
e ba
ck o
n th
e jo
b?
157
156
65
The
fin
ishe
d pr
ocL
z..-
t now
pro
duce
d ne
eds
to b
e ch
ecke
d fo
r fl
ow th
at is
a y
es a
nsw
er m
ust b
e gi
ven
to th
e si
x "f
inal
"
ques
tions
or
a cr
itica
l lin
k m
ay b
e m
issi
ng:
158
159
66
MA
TE
RIA
L D
EV
EL
OPM
EN
T
1.C
heck
the
stat
emen
t of
the
perf
orm
ance
pro
blem
aga
inst
the
corp
orat
e go
al.
_Is
the
prob
lem
wor
th s
olvi
ng?
2. C
ompa
re th
e sk
ill/k
now
ledg
e re
quir
emen
ts a
gain
st th
e pe
rfor
man
ce p
robl
em.
If y
our
targ
et p
erfo
rmer
s ac
quir
e th
ese
will
it e
limin
ate
the
prob
lem
?
3. C
ompa
re th
e ov
eral
l goa
l and
inst
ruct
iona
l obj
ectiv
es f
or th
e co
urse
aga
inst
the
skill
& k
now
ledg
e re
quir
emen
ts.
Are
the
obje
ctiv
es d
irec
tly r
elat
ed to
the
skill
& k
now
ledg
e?
4 C
ompa
re th
e po
stte
st a
gain
st th
e in
stru
ctio
nal o
bjec
tives
.
Doe
s th
e te
st m
easu
re th
e ac
hiev
emen
t of
thes
e ob
ject
ives
?
5. C
heck
the
cour
se c
onte
nts
agai
nst t
he f
inal
test
.
Will
thes
e co
nten
ts h
elp
the
stud
ent a
chie
ve a
hig
h sc
ore
on th
e po
stte
st?
Whe
n in
dou
bt th
row
it o
ut!
6. C
ompa
re th
e te
achi
ng-l
earn
ing
activ
ities
aga
inst
the
cour
se c
onte
nts
and
the
post
test
.
Will
thes
e ac
tiviti
es e
nabl
e th
e st
uden
ts to
pro
cess
the
cont
ent a
nd a
chie
ve a
hig
h sc
ore
on th
e po
stte
st?
161
160
67
Che
cklis
t of
Act
iviti
es d
urin
g M
ater
ials
Dev
elop
men
t
Coo
rdin
ate
App
ropr
iate
sup
port
per
sonn
el (
med
ia p
rodu
ctio
n sp
ecia
lists
and
cou
rse
wri
ters
) to
ens
ure
that
the
desi
gn s
peci
fica
tions
are
relia
bly
tran
slat
ed in
to in
stru
ctio
nal m
ater
ials
.
Mat
eria
ls d
evel
opm
ent a
ctiv
ities
by
supp
ort p
erso
nnel
.
Rev
iew
wal
k-th
roug
h by
sub
ject
-mat
ter
and
othe
r ex
pert
s.
Dev
elop
men
t tes
ting
of th
e pr
otot
ype
mat
eria
ls.
Fiel
d tr
ials
.
Plan
On
desi
gnin
g an
d co
nduc
ting
revi
ew w
alk-
thro
ugh
by s
ubje
ct-m
atte
r an
d ot
her
expe
rts.
On
desi
gnin
g an
d co
nduc
ting
deve
lopm
ent t
estin
g of
the
prot
otyp
e m
ater
ials
.
On
desi
gnin
g an
d co
nduc
ting
fiel
d tr
ials
.
Mon
itor
MD
act
iviti
es a
nd p
rovi
de f
eedb
ack.
Ana
lysi
s
Of
data
fro
m d
evel
opm
ent t
estin
g an
d fi
eld
tria
ls a
nd s
ugge
st s
uita
ble
revi
sion
s.
Coo
rdin
ate
ongo
ing
eval
uatio
n of
var
ious
pro
duct
s du
ring
this
pha
se.
Rev
ise
Rev
iew
eva
luat
ion
resu
lts a
nd e
nsur
e su
itabl
e re
visi
ons
are
mad
e. D
istr
ibut
e ev
alua
tion
resu
lts to
rel
evan
t peo
ple. 16
316
268
Inst
ruct
iona
l Mat
eria
ls R
evie
wPr
int 1.
Con
sist
ency
. Use
a c
onsi
sten
t for
mat
thro
ugho
ut th
e pr
inte
d bo
ok. E
ach
less
on o
r ch
apte
r sh
ould
hav
e th
e sa
me
form
at.
2. F
orm
at. U
se e
ither
a h
oriz
onta
l or
a ve
rtic
al f
orm
at f
or y
our
book
. Do
not m
ix th
em u
p so
that
the
stud
ent h
as to
turn
the
book
.
3. D
ivis
ions
. Kee
p ea
ch s
ectio
n (c
hapt
er o
r le
sson
) cl
earl
y id
entif
ied.
With
in e
ach
of th
ese
sect
ions
hav
e a
cons
iste
nt la
yout
for
diff
eren
t uni
ts (
e.g.
, obj
ectiv
es, p
revi
ews,
and
sum
mar
ies)
.
4. H
eadi
ngs.
Kee
p al
l hea
uing
leve
ls c
lear
and
con
sist
ent.
Use
hea
ding
s fr
eque
ntly
.
5. H
eade
rs. M
ake
sure
eac
h pa
ge h
as a
hea
der
.
6. F
oote
rs. P
rovi
de a
foo
ter
for
each
pag
e w
ith r
efer
ence
s if
any
and
pag
e nu
mbe
r.
7. P
ages
. Org
aniz
e th
e co
nten
ts o
f a
page
cle
arly
and
con
sist
ently
.
8. S
paci
ng. H
ave
a co
nsis
tent
spa
cing
for
wor
ds a
nd s
ente
nces
.
9. J
ustif
icat
ion
Avo
id ju
stif
ying
the
righ
t mar
gin.
A r
agge
d ri
ght m
argi
n Is
eas
ier
to r
ead
10. W
ord
brea
ks. A
void
hyp
hena
ting
wor
ds a
t the
end
of
the
line.
11
Page
bre
ak. D
eter
min
e th
e st
oppi
ng p
oint
at t
he b
otto
m o
f th
e pa
ge b
y th
e co
nten
t rat
her
than
by
layo
ut. A
void
bre
akin
gre
late
d m
ater
ials
at t
he b
otto
m o
f th
e pa
ge.
165
164
69
12. C
opya
bilit
y. U
se ty
pefa
ces
whi
ch w
ill n
ot d
eter
iora
te w
ith r
epea
ted
copy
ing.
13. M
argi
ns. L
eave
am
ple
mar
gins
for
eas
e of
bin
ding
and
for
not
es.
14. I
llust
ratio
ns. M
ake
sure
that
illu
stra
tions
sup
port
the
text
con
tent
and
are
rel
evan
t to
it.
15. P
lace
men
t. M
ake
sure
that
the
illus
trat
ion
is p
lace
d in
suc
h a
way
that
all
refe
renc
es to
it a
re o
n th
e sa
me
page
.
16. C
aptio
ns. M
ake
sure
eac
h ill
ustr
atio
n ha
s a
capt
ion
to id
entif
y an
d to
des
crib
e it.
17. G
raph
ics.
Con
side
r th
e us
e of
dia
gram
s an
d fl
owch
arts
inst
ead
of le
ngth
y pa
ragr
aphs
of
narr
ativ
e pr
ose
18 T
able
s. U
se ta
bles
whe
neve
r po
ssib
le to
pre
sent
info
rmat
ion
in a
n or
gani
zed
fash
ion.
19 T
able
pla
cem
ent.
Mak
e su
re th
e ta
ble
is p
lace
d in
an
area
whe
re a
ll of
the
acco
mpa
nyin
g te
xt is
ava
ilabl
e.
20. C
olor
. Avo
id u
sing
col
or b
ecau
se it
is e
xpen
sive
and
, exc
ept i
n so
me
spec
ial c
ases
, doe
s no
t con
trib
ute
to th
e in
stru
ctio
nal
effe
ctiv
enes
s.
21. B
indi
ng. O
rgan
ize
the
mat
eria
ls in
to ta
bbed
sec
tions
for
eas
e of
ref
eren
ce.
22. E
mph
asis
. Use
bol
d fa
ce, u
nder
linin
g, a
nd it
alic
s to
em
phas
ize
key
wor
ds. H
owev
er, a
void
thei
r ov
erus
e.
23. C
olor
ed p
ages
. Con
side
r th
e us
e of
col
ored
pap
er to
iden
tify
diff
eren
t sec
tions
of
the
book
.
167
166
70
Aud
io 1. L
angu
age.
Use
sim
plic
ity in
you
r sc
ript
. Am
ong
othe
r th
ings
, use
con
trac
tions
fre
quen
tly. B
ut r
efra
in f
rom
ext
ensi
ve u
se o
fac
rony
ms.
2. W
ords
. Avo
id te
chni
cal.
Kee
p ja
rgon
and
tech
nica
l ter
ms
to a
min
imum
.
3. V
oice
s. K
eep
the
num
ber
of c
hara
cter
s at
a m
inim
um.
4. S
ound
eff
ects
. Whe
neve
r ap
prop
riat
e, u
se s
ound
eff
ects
to a
dd r
ealis
m to
the
audi
otap
e.
5. M
usic
Use
mus
ic to
sig
nal l
ogic
al u
nits
of
thou
ght.
Mak
e su
re y
ou a
re n
ot v
iola
ting
copy
righ
t
law
s w
ith y
our
use
of m
usic
.
6. I
nter
activ
ely.
Pro
vide
opp
ortu
nitie
s fo
r, th
e st
uden
t to
activ
ely
part
icip
ate.
Do
not s
ound
as
if y
ou a
re le
ctur
ing
to a
larg
egr
oup,
but
try
to s
imul
ate
a co
nver
satio
n-w
ith a
n in
divi
dual
.
7. P
rint
sup
port
. If
the
audi
otap
e is
acc
ompa
nied
by
a w
orkb
ook,
mak
e su
re th
at th
e in
stru
ctio
ns o
n th
e ta
pe (
e.g.
, pag
ere
fere
nces
) ar
e ac
cura
te a
nd c
onsi
sten
t.
8. D
irec
tions
. If
the
tape
has
to b
e st
oppe
d w
hile
the
stud
ent w
orks
out
a p
robl
em, f
or e
xam
ple,
be
sure
to in
clud
e ap
prop
riat
edi
rect
ions
. Als
o, in
clud
e di
rect
ions
at t
he e
nd o
f on
e si
de o
f th
e ta
pe to
turn
it o
ver.
9. V
isua
l sup
port
. If
the
audi
otap
e is
acc
ompa
nied
by
a se
t of
slid
es, m
ake
sure
the
visu
al m
essa
ge o
n th
e sl
ides
and
the
audi
tory
mes
sage
on
the
tape
rei
nfor
ce e
ach
othe
r. A
void
dis
trac
ting
or o
verl
y re
dund
ant c
ombi
natio
ns.
169
168
71
10. V
isua
l cha
nges
. Mak
e su
re th
at s
lide
chan
ges
are
care
fully
coo
rdin
ated
with
the
audi
oscr
ipt.
Do
not l
eave
the
sam
e vi
sual
on th
e sc
reen
for
long
per
iods
of
time.
Typ
e
1.D
o no
t use
dec
orat
ive
styl
es. U
se f
amili
ar s
ans
seri
f ty
pe s
tyle
for
title
s an
d se
rif
for
body
text
.
2.T
ype
size
. Do
not u
se ty
pe s
ize
smal
ler
than
one
-fou
rth
inch
(4
poin
t).
3.A
lignm
ent.
Cen
ter
all t
itles
. Beg
in a
ll ot
her
text
at t
he le
ft m
argi
n.
4.C
apita
lizat
ion.
Use
all
caps
for
title
s an
d m
ajor
hea
ding
s. F
or th
e m
ain
body
, use
onl
y in
itial
cap
itals
.
5.Sp
acin
g. L
eave
mor
e sp
ace
betw
een
units
of
text
than
with
in u
nits
. For
exa
mpl
es, d
oubl
e6.
spac
e be
twee
n "p
arag
raph
s" a
nd s
ingl
e sp
ace
with
in p
arag
raph
s.
7.L
istin
g. U
se n
umbe
rs, l
ette
rs, o
r bu
llets
to id
entif
y ea
ch it
em o
n a
list.
Num
bers
are
bes
t, es
peci
ally
whe
n yo
u ha
ve to
ref
er to
diff
eren
t ite
ms.
Use
bul
lets
if y
ou w
ant t
o av
oid
sugg
estin
g an
y se
quen
ce in
you
r lis
t.
8.In
dent
atio
n. I
f yo
u ha
ve a
nes
ted
list (
i.e.,
a lis
t con
tain
ing
diff
eren
t lev
els)
use
inde
ntat
ion
to s
how
the
leve
ls.
Slid
es
1. N
eces
sity
. Is
the
slid
e re
ally
nec
essa
ry?
If y
ou h
ave
doub
ts, l
eave
it o
ut.
2Fo
rmat
. Alw
ays
use
the
hori
zont
al f
orm
at f
or s
lides
.
3. S
cale
. If
you
are
show
ing
an o
bjec
t whi
ch is
unf
amili
ar to
the
stud
ents
, inc
lude
som
e fa
mili
ar o
bjec
t in
the
sam
e sl
ide
to s
ugge
stth
e si
ze.
4. P
ersp
ectiv
e. I
f yo
u ar
e sh
owin
g eq
uipm
ent o
r to
ols,
sho
w it
fro
m th
e po
int o
f vi
ew o
f th
e st
uden
t.
171
1 /0
72
5. G
raph
ics.
If
you
are
usin
g a
grap
hic,
be
sure
It i
s si
mpl
e. A
void
all
unne
cess
ary
deco
ratio
ns.
6. L
egib
ility
. The
illu
stra
tions
and
the
text
in th
e sl
ide
shou
ld b
e le
gibl
e to
a s
tude
nt s
eate
d th
e la
st r
ow.
7. C
olor
. Mos
t stu
dent
s ex
pect
col
or s
lides
. Unl
ess
ther
e is
som
e sp
ecia
l rea
son,
avo
id b
lack
and
whi
te g
raph
ics
in y
our
slid
es
8. C
aptio
ns. U
se te
xt s
pari
ngly
. If
you
need
to u
se c
aptio
ns, u
se a
cle
ar a
nd s
impl
e ty
pefa
ce.
9. S
eque
nce.
Mak
e su
re th
e sl
ides
are
seq
uenc
ed in
the
prop
er o
rder
.
10. A
udio
visu
al c
ombi
natio
n. M
ost s
lides
are
acc
ompa
nied
by
an a
udio
tape
or
inst
ruct
or ta
lk. T
he a
udito
ry m
essa
ge s
houl
d no
tcl
ash
with
(or
be e
xtre
mel
y re
dund
ant w
ith)
the
visu
al m
essa
ge o
n th
e sl
ide.
173
172
73
How
To
Con
duct
a %
WA
-Thr
ough
1. P
lan
your
wal
k-th
roug
h ca
refu
lly to
obt
ain
usef
ul d
ata.
Inc
lude
in th
e pl
ande
tails
of
wha
t you
wan
t to
test
, and
wha
t typ
e of
data
you
nee
d. I
dent
ify
your
exp
erts
and
pre
pare
a s
ched
ule.
.Se
lect
the
stud
ents
on
the
basi
s of
the
info
rmat
ion
you
wan
t to
colle
ct.
3. U
se tw
o ex
pert
s fo
r ea
ch a
rea
of e
valu
atio
n. C
ompa
re th
eir
com
men
tsan
d de
cide
whi
ch o
nes
are
criti
cal
4. F
ocus
the
revi
ew o
f ea
ch e
xper
t by
givi
ng th
em a
cle
ar in
dica
tion
of th
e ty
peof
fee
dbac
k yo
u w
ant.
It is
not
eno
ugh
for
you
to p
lunk
dow
n a
set o
f m
ater
ials
and
say
, "I
wou
ld li
ke to
hav
e yo
ur c
omm
ents
."
5. Y
ou m
ay w
ant t
o pr
ovid
e th
e in
stru
ctio
nal o
bjec
tives
to s
ome
expe
rts
and
with
hold
them
fro
m o
ther
s. Y
ou s
houl
d be
inte
rest
ed in
fin
ding
out
the
reac
tions
of
peop
le w
ho a
re n
ot b
iase
d by
the
stat
emen
tsof
you
r in
tend
ed o
utco
mes
.
6. S
ugge
st to
the
expe
rts
that
they
mak
e su
itabl
e ch
ange
s on
the
mat
eria
ls. I
f th
is is
not
fea
sibl
e. a
sk th
em f
or s
peci
fic
solu
tions
rath
er th
an g
ener
al c
omm
ents
.
7. A
sk y
our
expe
rt to
giv
e th
e co
mm
ents
and
sug
gest
ions
inw
ritin
g. T
his
prov
ides
a u
sefu
l rec
ord
for
late
r re
view
.
8. C
ondu
ct a
deb
rief
ing
inte
rvie
w w
ith e
ach
expe
rt in
a f
ace-
to-f
ace
situ
atio
n or
ove
r th
e te
leph
one.
9. M
ake
the
expe
rts'
job
easi
er. S
ugge
st th
at th
ey m
ay d
icta
teth
eir
com
men
ts o
n a
cass
ette
tape
rat
h( r
than
wri
ting
them
dow
n.
1I
10.E
mph
asiz
e by
you
r w
ords
and
act
ions
that
you
take
the
wal
k th
roug
hse
riou
sly
and
not a
s a
polit
ical
exe
rcis
e Y
ou m
ay n
ot
be a
ble
to in
corp
orat
e al
l sug
gest
ions
, but
mak
e a
cons
ciou
s ef
fort
toco
nsid
er e
very
one
of
them
.
175
174
74
How
To
Con
duct
a D
evel
opm
enta
l Tes
t
BE
FOR
E Y
OU
R F
IRST
DE
VE
LO
PME
NT
AL
TE
STIN
G S
ESS
ION
:
1. C
lean
up
your
trai
ning
mat
eria
ls s
o th
at th
e st
uden
t is
not u
nnec
essa
rily
con
fuse
d. M
ake
sure
that
you
r pa
ges
are
not f
ull o
fty
pos
and
your
aud
iota
pe is
aud
ible
. Don
't st
rive
for
per
fect
ion
or b
ecom
e em
otio
nally
atta
ched
to y
our
prod
uct.
2 If
the
trai
ning
mat
eria
l doe
s no
t req
uire
the
stud
ent t
o m
ake
freq
uent
act
ive
resp
onse
s, p
repa
re a
set
of
rele
vant
que
stio
ns to
be in
ters
pers
ed a
t the
end
of
each
logi
cal u
nit.
3. P
repa
re a
cri
teri
on te
st to
mea
sure
the
trai
ning
obj
ectiv
es. I
f th
e m
ater
ial r
equi
res
spec
ific
ent
ry s
kills
and
kno
wle
dge,
prep
are
a pr
ereq
uisi
te te
st.
4. S
elec
t a s
mal
l gro
up (
five
to te
n) o
f st
uden
ts w
ho r
epre
sent
the
type
of
stud
ent f
or w
hom
the
trai
ning
mat
eria
l is
to b
ead
apte
d.
INIT
IAT
ING
TH
E D
EV
EL
OPM
EN
TA
L T
EST
ING
SE
SSIO
N:
6. B
egin
with
the
"sm
arte
r" a
nd m
ore
expr
essi
ve s
tude
nt T
ell y
our
grou
p an
d w
ork
with
him
or
her
indi
vidu
ally
.
7. M
ake
sure
that
the
stud
ent u
nder
stan
ds th
at it
is th
e m
ater
ial (
and
not h
im o
r he
r) th
at is
bei
ng e
valu
ated
and
that
you
are
ther
e to
obs
erve
and
not
8. D
o no
t aro
use
the
stud
ent's
anx
iety
leve
l by
form
ally
adm
inis
teri
ng th
e en
try
test
and
the
pret
est.
Info
rmal
ly in
terv
iew
the
stud
ent t
o fi
nd o
ut if
he
or s
he h
as th
e pr
ereq
uisi
te k
now
ledg
e. I
f un
sure
, use
sel
ecte
d pa
rts
of th
e en
try
test
. Als
o in
terv
iew
tofi
nd o
ut if
he
or s
he h
as a
lrea
dy a
chie
ved
som
e of
the
trai
ning
obj
ectiv
es. I
f un
sure
, use
sel
ecte
d pa
rts
of th
e cr
iteri
on te
st
9. H
ave
the
stud
ent w
orki
ng th
roug
h th
e m
ater
ial a
s so
on a
s po
ssib
le.
177
176
75
10. U
se th
e du
plic
ate
copy
or
scri
pt f
or r
ecor
ding
stu
dent
res
pons
es, r
emar
ks a
nd r
eact
ions
.
11. E
ncou
rage
and
rec
ord
stud
ent c
omm
ents
abo
ut le
vel.
Mak
e no
tes
of th
ese
rem
arks
at a
ppro
pria
te p
lace
s in
you
r du
plic
ate
copy
or
scri
pt.
12. E
ncou
rage
and
rec
ord
stud
ent c
omm
ents
abo
ut h
ow th
e m
ater
ial c
an b
e im
prov
ed.
113
Rec
ord
nonv
erba
l rea
ctio
ns o
f th
e st
uden
t whi
ch in
dica
te s
uch
thin
gs a
s co
nfus
ion,
fru
stra
tion,
and
bor
edom
. Bri
efly
pro
beth
e st
uden
t for
the
caus
e of
thes
e re
actio
ns if
it is
not
obv
ious
.
INT
ER
VE
NIN
G D
UR
ING
DE
VE
LO
PME
NT
AL
TE
STIN
G S
ESS
ION
S:
14. I
n sc
hedu
ling
your
ses
sion
, rem
embe
r to
allo
w ti
me
for
debr
iefi
ng a
ctiv
ities
. Try
to u
se th
e no
rmal
tim
e th
e st
uden
t wou
ldsp
end
wor
king
on
the
mat
eria
l und
er c
lass
room
con
ditio
ns. T
erm
inat
e th
e se
ssio
n be
fore
the
stud
ent g
ets
tired
.
15. I
f yo
u of
e te
rmin
atin
g th
e de
velo
pmen
tal t
estin
g se
ssio
n in
the
mid
dle,
mak
e su
re th
at y
ou d
o so
at t
he e
nd o
f so
me
logi
cal
unit
of in
stru
ctio
n.
16. I
nfor
mal
ly in
terv
iew
the
stud
ent t
o ch
eck
his
rete
ntio
n of
wha
t he
has
lear
ned
Use
cri
teri
on te
st it
ems
to s
truc
ture
this
info
rmal
17. D
ebri
ef th
e st
uden
t by
aski
ng h
im o
pen-
end
ques
tions
abo
ut th
e ef
fect
s of
the
trai
ning
mat
eria
l and
way
s to
impr
ove
it.
18 I
ncor
pora
te a
ll re
visi
ons
in th
e tr
aini
ng m
ater
ial a
s so
on a
s po
ssib
le a
fter
the
deve
lopm
enta
l tes
ting
sess
ion.
Cle
an u
p th
em
ater
ial f
or th
e ne
xt s
essi
on.
179
178
76
19. C
ondu
ct in
divi
dual
tryo
uts
with
new
stu
dent
s to
che
ck th
e re
vise
d ve
rsio
n. U
se th
e sa
me
proc
edur
e as
bef
ore.
Con
tinue
deve
lopm
enta
l tes
ting
sess
ions
unt
il th
e tr
aini
ng m
ater
ial b
egin
s to
pro
duce
con
sist
ent a
nd s
atis
fact
ory
resu
lts. A
t thi
s tim
em
ake
all n
eces
sary
rev
isio
ns a
nd p
rodu
ce m
ultip
le c
opie
s of
the
mat
eria
l.
SEC
ON
D P
HA
SE O
F D
EV
EL
OPM
EN
TA
L T
EST
ING
:
1. T
est t
he r
evis
ed tr
aini
ng m
ater
ial w
ith a
gro
up o
f st
uden
ts u
nder
act
ual c
lass
room
con
ditio
ns.
2.C
olle
ct s
tude
m r
espo
nses
to q
uest
ions
whi
ch a
re in
ters
pers
ed w
ithin
the
trai
ning
mat
eria
l and
in th
e cr
iteri
on te
st.
3. C
olle
ct s
tude
nt f
eedb
ack
on a
ttitu
de to
war
d th
e tr
aini
ng m
ater
ial a
nd s
ugge
stio
ns f
or
impr
ovem
ent.
4. C
onst
ruct
a m
atri
x of
stu
dent
res
pons
es to
dif
fere
nt q
uest
ions
. Ins
pect
this
mat
rix
to id
entif
y
erro
r pa
ttern
s w
hich
sug
gest
sec
tions
to b
e re
vise
d.
5. M
ake
suita
ble
chan
ges
in th
e tr
aini
ng m
ater
ials
to r
educ
e or
elim
inat
e pr
oble
m a
reas
.
CO
NT
INU
ED
DE
VE
LO
PME
NT
AL
TE
STIN
G:
6. K
eep
a m
aste
r co
py o
f th
e tr
aini
ng m
ater
ials
. Not
e do
wn
stud
ent p
robl
ems
and
sugg
estio
ns f
orim
prov
emen
t and
whe
n th
ey o
ccur
as
you
use
the
mat
eria
l in
your
cla
ssro
om.
7. P
erio
dica
lly r
evie
w y
our
accu
mul
ated
stu
dent
ver
ific
atio
n no
tes
and
mak
e ap
prop
riat
e ch
ange
s.
RE
VIS
ION
S:
0
181
77
8. M
aint
ain
suff
icie
nt e
mot
iona
l det
achm
ent f
rom
you
r tr
aini
ng m
ater
ial.
If s
tude
nt v
erif
icat
ion
feed
back
con
sist
ently
sho
ws
the
mat
eria
l to
be in
effe
ctiv
e, c
onsi
der
abor
ting
the
proj
ect a
nd
star
ting
from
scr
atch
. Don
't th
row
aw
ay g
ood
mon
ey a
nd e
ffor
t try
ing
to s
ave
usel
ess
mat
eria
ls.
9. D
on't
mak
e an
y ch
ange
s in
the
trai
ning
mat
eria
ls w
ithou
t jus
tifia
ble
caus
e. M
akin
g co
smet
ic
chan
ges
may
act
ually
red
uce
the
effe
ctiv
enes
s of
the
mat
eria
ls.
10 M
ore
teac
hing
and
telli
ng a
re n
ot a
lway
s th
e be
st w
ay to
impr
ove
the
trai
ning
eff
ectiv
enes
s of
the
mat
eria
l Whe
n st
uden
ts h
ave
prob
lem
s in
rec
allin
g th
e m
ain
idea
or
in a
pply
ing
a ru
le, y
oum
ay w
ant t
o se
arch
for
irre
leva
nt c
onte
nt a
nd a
ctiv
ities
and
elim
inat
e th
em. B
e es
peci
ally
ruth
less
with
cut
e pi
ctur
es w
hich
are
sup
pose
d to
enr
ich.
183
182
78
Rev
isio
ns B
ased
on
Dev
elop
men
tal T
est F
eedb
ack
IF.
Stud
ents
hav
e pr
oble
ms
with
con
cept
s.
Stud
ents
hav
e pr
oble
ms
in d
iscr
imin
atio
n(m
ista
ke n
onex
ampl
es f
or e
xam
ples
)
Stud
ents
rec
all t
he w
ords
but
can
not
appl
y a
rule
Stud
ents
ans
wer
que
stio
ns w
hile
wor
king
thro
ugh
the
mat
eria
l, bu
t fai
l in
the
fina
l tes
t
Stud
ents
com
e ba
ck to
che
ck w
ith y
ou o
nth
e co
rrec
tnes
s of
thei
r an
swer
s or
do
not
unde
rsta
nd w
hy th
eir
answ
er is
inco
rrec
t
184
TH
EN
con
side
r th
is.
Use
a n
umbe
r of
con
cret
e ex
ampl
es. C
hoos
e ex
ampl
es th
at a
re c
lear
-cut
and
uncl
utte
red
prin
cipl
es, a
nd r
ules
to in
trod
uce
the
conc
ept.
Mak
e su
re th
atth
ese
exam
ples
are
mea
ning
ful t
o th
e st
uden
ts. S
tren
gthe
n m
aste
ry w
ith m
ore
com
plex
exa
mpl
es la
ter.
Pres
ent a
num
ber
of n
onex
ampl
es a
nd p
oint
out
wha
t cri
tical
attr
ibut
es th
eyla
ck .B
egin
with
obv
ious
non
exam
ples
and
mov
e to
war
d su
btle
r on
es th
atre
quir
e fi
ner
disc
rim
inat
ions
.
Prov
ide
mor
e op
port
uniti
es f
or a
ctiv
e pa
rtic
ipat
ion,
. Bui
ld a
var
iety
of
ques
tions
and
app
licat
ion
exer
cise
s in
the
mat
eria
l.
Inco
rpor
ate
freq
uent
rev
iew
s an
d in
crea
se th
e nu
mbe
r of
opp
ortu
nitie
s fo
rac
tive
part
icip
atio
n
Prov
ide
built
-in
feed
back
in th
e m
ater
ial.
Hav
e fe
edba
ck s
tate
men
ts g
ive
diag
nost
ic in
form
atio
n ra
ther
than
mer
ely
telli
ng th
e st
uden
t if
he o
r sh
e is
corr
ect o
r in
corr
ect.
185
79
Rev
isio
ns B
ased
on
Dev
elop
men
tal
Tes
t Fee
dbac
k
IF .
Stud
ents
rec
all i
ndiv
idua
l ste
psin
a
proc
edur
e bu
t can
not a
pply
the
entir
epr
oced
ure
in th
e ri
ght o
rder
Stud
ent p
rovi
de c
orre
ct a
nsw
ers
in th
e
fina
l tes
t but
can
not u
se th
eir
skill
s/kn
owle
dge
at a
late
r tim
e
Stud
ents
are
con
stan
tlygu
essi
ng o
n
answ
ers
toqu
estio
ns
Stud
ents
con
sist
ently
fai
l to
achi
eve
the
obje
ctiv
es o
f th
e tr
aini
ngm
ater
ials
Stud
ents
hav
e pr
oble
ms
unde
rsta
ndin
g a
conc
ept
TH
EN
con
side
r th
is
You
are
pro
babl
y di
stra
ctin
gth
em w
ith e
xerc
ises
in e
ach
step
.
Beg
in w
ith a
n ex
ampl
e of
com
plet
e ap
plic
atio
n. P
rese
ntla
rger
chu
nks
of
step
s be
fore
ask
ing
the
stud
ent t
o ap
ply
them
.Pr
ovid
e ad
ditio
nal e
xerc
ises
in
reca
lling
the
sequ
ence
.
Del
iber
atel
y bu
ild u
p an
app
licat
ion
unit.
Use
a w
ide
vari
ety
ofex
ampl
es to
dem
onst
rate
the
tran
sfer
of
the
skill
s an
d kn
owle
dge
to o
ther
curr
icul
um a
reas
and
to r
eal-
life
situ
atio
ns.
Prov
ide
addi
tiona
l ins
truc
tion.
Use
inco
rrec
t gue
sses
to id
entif
yth
e ty
pes
of
stud
ent m
isco
ncep
tions
and
prov
ide
suita
ble
rem
edia
lin
stru
ctio
n.
It is
pos
sibl
e th
at y
ou a
reat
tem
ptin
g to
o bi
g a
jum
p to
o so
on.
sim
plif
ying
your
obj
ectiv
e is
ale
gitim
ate
revi
sion
. It i
s be
tter
for
mor
e st
uden
ts to
ach
ieve
low
er-l
evel
obj
ectiv
es th
an f
or n
ost
uden
t to
achi
eve
your
high
er-l
evel
one
s.
Che
ck th
e le
vel o
f yo
urla
ngua
ge. U
nles
s ab
solu
tely
requ
ired
by
the
trai
ning
obje
ctiv
es, a
void
dif
ficu
lt w
orts
and
com
plex
sen
tenc
e pa
ttern
s.E
ven
if y
our
obje
ctiv
es c
all f
or th
e m
aste
ryof
unc
omm
on v
ocab
ular
yan
d co
mpl
ex
sent
ence
s, s
tart
with
sim
ple
and
easy
lang
uage
.
Stud
ents
hav
e di
ffic
ulty
reca
lling
ear
lier
info
rmat
ion
Con
side
r di
vidi
ng in
stru
ctio
nin
to s
mal
ler
units
. Als
o us
e a
suita
ble
syst
em
of ti
tles
and
head
ings
whi
ch p
rovi
de s
truc
tura
l hoo
ksfo
r th
e re
cal:
of
info
rmat
ion.
Stud
ents
hav
e pr
oble
ms
reca
lling
info
rmat
ion
See
if r
eseq
uenc
ing
will
hel
p.A
mor
e lo
gica
l or
deve
lopm
enta
l org
aniz
atio
n
can
impr
ove
the
mas
tery
of
the
cont
ent
1f18
781
)
Rev
isio
ns B
ased
on
Dev
elop
men
tal T
est F
eedb
ack
IF.
TH
EN
con
side
r th
is
Stud
ents
fee
l fru
stra
ted
from
the
begi
nnin
g
Stud
ents
kee
p co
min
g ba
ck to
you
with
ques
tions
Stud
er).
- ha
ve p
robl
ems
unde
rsta
ndin
gth
e tr
aini
ng g
oals
Stud
ents
are
con
fuse
d ab
out s
ome
of th
eco
ncep
ts
188
The
y m
ay la
ck s
ome
prer
equi
site
ski
lls a
nd k
now
ledg
e. C
heck
out
this
hyp
othe
sis
If tr
ue, p
rodu
ce b
ridg
ing
mat
eria
ls to
pre
cede
you
rtr
aini
ng m
ater
ial.
Perh
aps
the
trai
ning
mat
eria
ls m
ay n
ot b
e su
itabl
e fo
r us
e in
a s
elf-
trai
ning
mod
e. A
n ef
fici
ent w
ay o
f re
visi
ng th
e m
ater
ial i
s to
inco
rpor
ate
it w
ithin
atu
tori
ng s
et-u
p. T
his
requ
ires
you
to p
repa
re tr
aini
ng m
ater
ials
for
pee
r or
para
prof
essi
onal
tuto
rs to
com
pens
ate
for
the
defi
cien
cies
of
the
mat
eria
l.
Beg
in w
ith a
pre
view
or
adva
nce
orga
nize
r. U
se n
onte
chni
cal l
angu
age
tobr
iefl
y pr
esen
t wha
t the
stu
dent
will
be
able
to d
o up
onco
mpl
etio
n of
the
trai
ning
mat
eria
l.
Try
illu
stra
tions
and
vis
uals
. A s
ingl
e di
agra
m m
aypr
esen
t the
str
uctu
re o
f a
conc
ept m
uch
mor
e cl
earl
yth
an te
n th
ousa
nd w
ords
.
189
81
Mat
eria
ls D
evel
opm
enic
Eva
luat
ion
Qua
lity Fina
l mat
eria
ls
Subj
ect-
mat
ter
.-
*Dev
elop
men
tal t
estin
gW
alk-
thro
ugh
proc
ess
Fiel
d te
stpr
oces
sSt
uden
ts f
or te
stin
g
70
*Dat
a co
llect
ion
BE
ST
CO
PY
AV
AIL
AB
LE19
1
Mat
eria
ls D
evel
opm
ent E
valu
atio
n
Form
ativ
e C
rite
ria
Que
stio
nsR
atin
g an
d C
omm
ents
LO
WH
IGH
12
34
5
Prod
uctio
n qu
ality
Inst
ruct
iona
l qua
lity
Subj
ect-
mat
ter
qual
ity
Wal
k-th
roug
h pr
oces
s
Dev
elop
men
tal t
estin
gpr
oces
s
192
Do
the
fina
l mat
eria
ls d
emon
stra
te
high
pro
duct
ion
qual
ity in
the
sele
cted
inst
ruct
iona
l med
ia a
nd
deliv
ery
stra
tegy
?
Do
the
fina
l mat
eria
ls r
efle
ct
appr
opri
ate
appl
icat
ion
ofin
stru
ctio
nal d
esig
n pr
inci
ples
?
Is th
e co
nten
t pre
sent
ed in
the
cour
se a
ccur
ate,
app
ropr
iate
, and
up-t
o-da
te?
Wer
e th
e dr
aft m
ater
ials
rev
iew
ed b
yap
prop
riat
e ex
pert
s an
d re
vise
dac
cord
ing
to th
eir
feed
back
?
Wer
e th
e pr
otot
ype
inst
ruct
iona
lm
ater
ials
sub
ject
ed to
indi
vidu
al o
rsm
all g
roup
dev
elop
men
tal t
ests
and
revi
sed
on th
e ba
sis
of th
e fe
edba
ck?
193
82
Mat
eria
ls D
evel
opm
ent E
valu
atio
n
Form
ativ
e C
rite
ria
uest
ions
Fiel
d te
st p
roce
ss
Stud
ents
for
test
ing
Dat
a co
llect
ion
194
Wer
e th
e ne
ar f
inal
ver
sion
s of
the
mat
eria
l sub
ject
ed to
an
appr
opri
ate
test
und
er f
ield
con
ditio
ns a
ndre
vise
d on
the
basi
s of
the
find
ings
?
Are
the
deve
lopm
enta
l/fie
ld te
stst
uden
ts r
epre
sent
ativ
e of
the
targ
etpo
pula
tion?
Has
the
deve
lopm
enta
Ufi
eld
test
been
pla
nned
ade
quat
ely
to c
olle
ctre
liabl
e an
d va
lid d
ata?
Hav
e da
taco
llect
ion
form
s, in
stru
men
ts, a
ndeq
uipm
ent b
een
appr
opri
atel
ypr
epar
ed?
Rat
ing
and
Com
men
tsL
OW
HIG
H1
23
45
195
83
"M3,:z.o,
Vid
eo R
evie
w
exce
llent
PAR
T I
. OV
ER
AL
L R
EV
IEW
poor
1. I
s th
e pr
oble
m c
lear
ly d
efin
ed?
12
34
5
2. S
tate
men
t of
obje
ctiv
es. (
Are
the
cour
se o
bjec
tives
sta
ted
in a
nac
cept
able
for
mat
?)1
23
45
3. T
ests
(or
test
spe
cifi
catio
ns).
(D
o th
e te
sts
or th
eir
spec
ific
atio
nsdi
rect
ly m
easu
re th
e ac
hiev
emen
t of
all i
nstr
uctio
nal o
bjec
tives
?)1
23
45
4. W
as a
Lea
rner
ana
lysi
s pr
oper
ly c
ondu
cted
?1
23
45
5. D
oes
the
Tra
inin
g N
eed
anal
ysis
incl
ude
a de
taile
d lis
ting
of1
23
45
Skill
s an
d K
now
ledg
e N
eces
sary
?
6. I
s th
e St
ory-
boar
d cl
ear
and
conc
ise
and
wel
l org
aniz
ed?
12
34
5
7. I
s th
e Sc
ript
cle
ar a
nd c
onci
se a
nd w
ell o
rgan
ized
?1
23
45
8. D
oes
the
vide
o te
ll a
logi
cal,
belie
vabl
e an
d re
leva
nt s
tory
?1
23
45
9. D
iffi
culty
leve
l. (I
s th
e di
ffic
ulty
leve
l of
the
cour
se s
uite
dto
the
targ
et p
opul
atio
n ra
ther
than
bei
ng to
o ha
rd o
r to
o ea
sy'?
)1
23
45
10. L
earn
ing
activ
ities
(D
oes
the
cour
se in
clud
e ap
prop
riat
ele
arni
ng a
ctiv
ities
for
dif
fere
nt e
vent
s of
inst
ruct
ion'
?)1
23
45
19B
199
84
Vid
eo R
evie
w
PAR
T I
I. F
ilm D
esig
n V
aria
bles
I.C
lari
ty o
f co
nten
t. M
eani
ng a
nd O
rgan
izat
ion
Sim
ple,
suc
cinc
t ver
bal c
omm
enta
ry. D
o no
t con
fuse
or
inun
date
the
perc
eptu
al f
ield
s us
ed in
lear
ning
Wat
ch e
labo
rate
atte
ntio
n ga
inin
g de
vise
s or
ove
r ex
tens
ive
use
of o
ptic
al e
ffec
ts.
No
extr
aneo
us d
evic
es a
nd e
ffec
ts.
2. R
ate
of D
evel
opm
ent.
Freq
uenc
y an
d/or
rat
e of
pro
duci
ng f
acts
.
Rat
e sh
ould
be
chal
leng
ing
but n
ot to
o sl
ow, r
ate
shou
ld b
e ai
med
at t
he m
edia
n le
arne
r.U
se p
revi
ous
lear
ner
anal
ysis
to d
eter
min
e ra
te f
or e
ach
"cha
nce'
of
info
rmat
ion.
Con
side
r w
hat l
earn
ers
will
be
doin
g w
hile
vie
win
g th
e in
form
atio
n; i.
e. li
sten
ing,
wri
ting,
rol
e-pl
ay.
Film
leng
th v
s. c
onte
nt, k
eep
cont
ent t
o m
anag
eabl
e am
ount
but
spe
ed th
e pr
ogra
m to
get
con
tent
acr
oss
if n
eces
sary
.
3. R
epet
ition
and
Fre
quen
cy. C
an h
ave
a si
gnif
ican
t im
pact
on
lear
ning
but
mus
t be
mon
itore
d fo
r fa
tigue
and
mon
oton
y.
Sum
mar
ies
sign
ific
ant c
hunk
s an
d co
nclu
sion
Dec
lara
tive
Stat
emen
ts w
ith I
nser
ted
ques
tions
Tes
ting
Pre,
Pos
t and
Em
bedd
edW
ritte
n st
udy
guid
es a
nd o
utlin
esSi
mul
tane
ous
visu
al a
nd a
udito
ry m
essa
ges
Lea
rner
con
trol
the
num
ber
of r
epet
ition
s th
ey w
ould
like
201
u0
85
Vid
eo R
evie
w
4. P
ract
ice
and
Feed
back
. Mak
e yo
ur p
rogr
am a
ctiv
e by
ask
ing
ques
tions
, usi
ng r
ole
play
and
sim
ulat
ion
Splic
e in
que
stio
ns e
tc. a
nd s
top
the
prog
ram
for
set
tim
e to
allo
w p
artic
ipan
t to
form
ulat
e re
spon
seL
eave
film
, pro
gram
ope
n en
ded
allo
w d
iscu
ssio
n at
end
to c
ompl
ete
the
pass
age.
5. P
artic
ipat
ion
and
Invo
lvem
ent
.Pr
ogra
m s
houl
d be
pro
duce
d ju
st a
s if
the
lear
ner
was
ther
e in
the
actio
n.
Shou
ld b
e fr
om th
e vi
ewer
s pe
rspe
ctiv
eU
se s
cena
rio
that
is f
amili
ar to
lear
ner
and
one
that
he
can
rela
te to
Use
cha
ract
ers
that
will
be
judg
ed a
s pr
estig
ious
Ass
ign
a ch
arac
ter
to le
arne
r or
gro
ups
of le
arne
rs d
iscu
ss th
e ro
le a
t con
clus
ion.
203
202
86
Mat
eria
ls D
evel
opm
ent S
hort
cuts
IU
se m
ore
wal
k-th
roug
h an
d le
ss f
ield
test
ing.
2.C
ondu
ct f
ewer
dev
elop
men
tal t
ests
and
fie
ld te
sts.
3.A
ct a
s th
e fi
eld
test
inst
ruct
or y
ours
elf
4.D
o no
t inc
lude
all
subg
roup
s in
the
fiel
d te
st.
5.R
elax
fie
ld c
ondi
tion
requ
irem
ents
, if
nece
ssar
y.
6.D
o no
t mea
sure
trai
nee
reac
tions
to c
ours
e.
7R
ecor
d da
ta o
nly
on m
ajor
cha
nges
to b
e m
ade.
8.A
llow
inst
ruct
ors
to c
orre
ct m
inor
pro
blem
s.
205
204
87
IMPL
EM
EN
TA
TIO
N
Plan
ning
for
the
pilo
t tes
ting
of th
e co
urse
sho
uld
occu
r as
ear
ly a
s po
ssib
le in
the
trai
ning
dev
elop
men
t pro
cess
.
Res
pons
ibili
ty f
or a
ll ne
cess
ary
envi
ronm
enta
l ite
ms
such
as
room
and
equ
ipm
ent n
eed
to b
e de
term
ined
and
doc
umen
ted.
Sche
dulin
g of
the
initi
al o
ffer
ing
and
freq
uenc
y an
d du
ratio
n of
cla
sses
onc
e th
e co
urse
is b
roug
ht o
n-st
ream
als
o ne
ed to
be
agre
ed to
. The
man
ager
sho
uld
wor
k cl
osel
y w
ith th
e cl
ient
,SM
E's
and
oth
ers
that
nee
d to
be
pres
ent i
n or
der
to a
ssur
e th
e
cour
se c
an b
e de
liver
ed im
med
iate
ly u
pon
com
plet
ion.
The
pre
dom
inan
t par
t of
the
audi
ence
sho
uld
be m
embe
rs o
f th
e ta
rget
audi
ence
. The
teac
her/
inst
ruct
or s
houl
d al
so b
e pr
esen
t for
the
initi
al o
ffer
ing
in o
rder
to g
athe
r re
al ti
me
feed
back
fro
m th
e
part
icip
ants
and
doc
umen
t any
cha
nges
that
may
be
need
ed. D
urin
g im
plem
enta
tion
clue
s to
look
for
incl
ude
(Abe
lla 1
986.
) :
1. P
artic
ipan
ts ta
lkin
g or
not
talk
ing
abou
t the
pro
gram
. 2. N
on-p
artic
ipan
ts n
ot b
eing
enc
oura
ged
to jo
in in
con
vers
atio
ns. 3
.
Man
y cy
nica
l rem
arks
. 4. P
artic
ipan
ts n
ot d
oing
ass
ignm
ents
. 5. P
artic
ipan
ts a
re la
te a
t sta
rt o
r br
eaks
or
skip
out
alto
geth
er.
6. P
artic
ipan
ts d
o no
t ask
for
hel
p or
add
ition
al in
form
atio
n or
idea
s. 7
. Hum
orou
s re
mar
ks m
ade
in c
lass
that
hav
e no
con
nect
ion
to c
lass
act
iviti
es. A
ll th
ese
may
be
sign
s of
impe
ndin
g pr
oble
ms
that
nee
d fu
rthe
r in
vest
igat
ion
adju
stm
ent.
The
adj
ustm
ents
havi
ng b
een
mad
e it
is n
ow ti
me
for
the
full
rele
ase
of th
e pr
ogra
m. T
his
shou
ld o
nly
be d
one
whe
n th
e fo
llow
ing
have
bee
n
answ
ered
to th
e cl
ient
s an
d yo
ur s
atis
fact
ion:
1 W
as th
e or
igin
al n
eed
accu
rate
ly d
efin
ed?
2 W
ere
the
obje
ctiv
es a
ppro
pria
te to
mee
t the
nee
d?
3 D
oes
the
prog
ram
des
ign
achi
eve
the
obje
ctiv
e?20
7
206
88
4. I
s th
e co
nten
t app
ropr
iate
to th
e ob
ject
ive?
5. A
re th
e m
ater
ials
of
good
qua
lity?
inte
rest
ing
and
wel
l wri
tten?
6. D
o th
e m
ater
ials
do
wha
t the
des
ign
inte
nds
them
to d
o?
208
209
89
Che
cklis
t of
Act
iviti
es d
urin
g Im
plem
enta
tion
Spec
ify
requ
irem
ents
For
inst
ruct
ors
or a
dmin
istr
ator
s of
the
cour
se.
Tra
inin
g fa
cilit
ies
and
equi
pmen
t.
Secu
re Suita
ble
inst
ruct
iona
l sta
ff.
Suita
ble
adm
inis
trat
ors
for
self
-pac
ed c
ours
es.
Coo
rdin
ate
Sche
dule
trai
ning
to m
eet o
rgan
izat
iona
l nee
ds.
Mon
itor
impl
emen
tatio
n of
the
cour
se w
ith c
ount
erpa
rts
in o
ther
orga
niza
tions
Eva
luat
ion.
Plan
for
sui
tabl
e fo
llow
-up
eval
uatio
n.
Prep
are
stra
tegi
es a
nd in
stru
men
tsfo
r da
ta c
olle
ctio
n du
ring
fol
low
-up
eval
uatio
n.
Ana
lyze
dat
a fr
om f
ollo
w-u
p ev
alua
tion.
Prep
are
follo
w-u
p ev
alua
tion
repo
rt.
21.1
2d)
90
Inst
ruct
iona
l Tec
hnol
ogy
Rev
iew
OV
ER
AL
L R
EV
IEW
Del
iver
y St
rate
gy s
ee jo
b ai
d "T
rain
ing
Del
iver
y st
rate
gies
"St
atem
ent o
f O
bjec
tives
Tes
ting
Lev
el 2
*Cou
rse
Con
tent
Met
hods
Med
ia
*Lea
rnin
g A
ctiv
ities
*.
212
Inst
ruct
iona
l Tec
hnol
ogy
Rev
iew
PAR
T I
. OV
ER
AL
L R
EV
IEW
1D
eliv
ery
stra
tegy
. (Is
the
deliv
ery
stra
tegy
the
mos
t cos
t-ef
fect
ive
one?
)
2. S
tate
men
t of
obje
ctiv
es. (
Are
the
cour
se o
bjec
tives
sta
ted
inan
acce
ptab
le f
orm
at?)
3. T
ests
(or
test
spe
cifi
catio
ns).
(D
o th
e te
sts
or th
eir
spec
ific
atio
nsdi
rect
ly m
easu
re th
e ac
hiev
emen
t of
all i
nstr
uctio
nal o
bjec
tives
?)
4 C
ours
e co
nten
t. (D
oes
the
cour
se p
rese
nt a
ll cr
itica
l inf
orm
atio
nre
late
d to
the
obje
ctiv
es r
athe
r th
an n
ice-
to-k
now
, sup
erfl
uous
, or
triv
ial f
acts
?)
5. I
nstr
uctio
nal m
edia
. (D
oes
the
cour
se u
se th
e m
ost c
ost-
effe
ctiv
eco
mbi
natio
n of
med
ia?)
6. I
nstr
uctio
nal m
etho
d. (
Doe
s th
e co
urse
use
the
mos
t cos
t-ef
fect
ive
com
bina
tion
of in
stru
ctio
nal m
etho
ds?)
7. L
earn
ing
activ
ities
. (D
oes
the
cour
se in
clud
e ap
prop
riat
e le
arni
ngac
tiviti
es f
or d
iffe
rent
eve
nts
of in
stru
ctio
n?)
8. P
rodu
ctio
n qu
ality
. (D
o th
e m
edia
ach
ieve
hig
h pr
oduc
tion
stan
dard
s?)
poor
exce
llent
12
34
5
12
34
5
12
34
5
12
34
5
12
34
5
12
34
5
12
34
5
12
34
5
21.4
215
Inst
ruct
iona
l Tec
hnol
ogy
Rev
iew
Part
II
Con
grue
ncy
Che
ck*
Inpu
tsPr
oced
ure
*Out
puts
Will
the
activ
ities
ena
ble
the
stud
ent t
opr
oces
sth
e co
nten
t and
ach
ieve
the
obje
ctiv
ein
the
mos
t eff
icie
nt a
nd e
ffec
tive
man
ner
poss
ible
?
216
217
9. D
iffi
culty
leve
l. (I
s th
e di
ffic
ulty
leve
l of
the
cour
se s
uite
dto
the
targ
et p
opul
atio
n ra
ther
than
bei
ng to
o ha
rd o
r to
o ea
sy?)
10 P
ract
ice
and
revi
ew. (
Doe
s th
e co
urse
pro
vide
app
ropr
iate
oppo
rtun
ities
for
the
prac
tice
and
revi
ew o
f cr
itica
l ski
lls a
ndkn
owle
dge?
)
11. F
eedb
ack
to s
tude
nts.
(D
oes
the
cour
se h
ave
prov
isio
ns f
orim
med
iate
fee
dbac
k to
stu
dent
s af
ter
they
com
plet
e va
riou
sex
erci
ses
and
test
s?)
12. T
rans
fer
and
appl
icat
ion.
(D
oes
the
cour
se e
mph
asiz
e th
eap
plic
atio
n of
ski
lls a
nd k
now
ledg
e to
job/
task
per
form
ance
?)
12
34
5
12
34
5
12
34
5
12
34
5
PAR
T H
. CO
NG
RU
EN
CY
CH
EC
K
1. B
rief
ly d
escr
ibe
the
perf
orm
ance
pro
blem
whi
ch y
our
cour
se is
des
igne
d to
sol
ve.
2. B
rief
ly d
escr
ibe
the
skilU
know
ledg
e re
quir
emen
ts f
or y
our
cour
se.
3. S
umm
ariz
e th
e ov
eral
l ins
truc
tiona
l obj
ectiv
es f
or y
our
cour
se.
4. O
btai
n a
copy
of
the
post
test
for
you
r co
urse
.
5. C
olle
ct a
ll th
e m
ater
ials
that
pro
ide
the
cont
ent f
or th
e co
urse
.
6. L
ist a
ll th
e im
port
ant t
each
ing-
lear
ning
act
iviti
es in
you
r co
urse
.
219
218
92
HO
W T
O D
O T
HE
CO
NG
RU
EN
CY
CH
EC
K:
I. C
heck
you
r st
atem
ent o
f th
e pe
rfor
man
ce p
robl
em a
gain
st th
e co
rpor
ate
mis
sion
and
goa
ls
Is th
is p
erfo
rman
ce p
robl
em w
orth
sol
ving
?
2. C
ompa
re y
our
skill
/kno
wle
dge
requ
irem
ents
aga
inst
the
perf
orm
ance
pro
blem
.
If y
our
targ
et p
erfo
rmer
s ac
quir
e th
ese
skill
s an
d kn
owle
dge,
will
it e
limin
ate
the
perf
orm
ance
pro
blem
?
3. C
ompa
re y
our
over
all g
oal f
or th
e co
urse
(an
d ot
her
inst
ruct
iona
l obj
ectiv
es)
agai
nst y
our
skill
/kno
wle
dge
requ
irem
ents
.
Are
the
obje
ctiv
es d
irec
tly r
elat
ed to
the
skill
s an
d kn
owle
dge?
4. C
ompa
re y
our
post
test
aga
inst
the
inst
ruct
iona
l obj
ectiv
es f
or th
e co
urse
.
Doe
s th
e te
st m
easu
re th
e ac
hiev
emen
t of
thes
e in
stru
ctio
nal o
bjec
tives
?
5. C
heck
the
cour
se c
onte
nts
agai
nst t
he f
inal
test
.
Will
thes
e co
nten
ts h
elp
the
stud
ent a
chie
ve a
hig
h sc
ore
on th
e po
stte
st?
6. C
ompa
re y
our
teac
hing
-lea
rnin
g ac
tiviti
es a
gain
st th
e co
urse
con
tent
s an
d th
e po
stte
st
221
24,0
93
Man
ki*
MM
?A
ssk:
,
ngsr
..MV
Me2
4§=
41.,a
n.
How
To
Intr
oduc
ea
Cou
rse
O F
ocus
b444
9::A
l
O P
lann
ing T
eam
wor
kM
ater
ials
Inst
ruct
or
O L
ocat
ion
Stud
ent s
elec
tion
O M
onito
ring
& R
evis
ion
, 7,7
,, ...
444:
016:
144:
::::e
t
How
To
Intr
oduc
e a
Cou
rse
I. F
ocus
. As
you
plan
for
the
inau
gura
l off
erin
g of
the
cour
se, r
emem
ber
the
purp
ose
of th
e ac
tivity
:
- T
o fa
mili
ariz
epo
tent
ial u
sers
of
the
cour
se w
ith it
s fe
atur
es.
- T
o pr
ovid
ein
itial
trai
ning
for
thos
e re
spon
sibl
e fo
r im
plem
entin
g th
e co
urse
.
2. P
lann
ing.
Pla
n fo
r th
e in
trod
uctio
n of
the
cour
se a
s ea
rly
in th
e tr
aini
ng d
evel
opm
ent p
roce
ss a
s po
ssib
le. (
Such
ear
lypl
anni
ng w
ill e
nsur
e th
at th
ere
is n
o si
gnif
ican
t del
ay b
etw
een
the
fiel
d te
st a
nd th
e im
plem
enta
tion.
)
3. T
eam
wor
k. W
ork
with
you
r cl
ient
, Tra
inin
g M
anag
er, a
nd T
rain
ing
Del
iver
y pe
rson
nel t
o pl
an f
or -
the
intr
oduc
tion
of y
our
cour
se.
4. D
etai
ls. W
ith o
ther
mem
bers
of
the
impl
emen
tatio
n te
am, w
ork
out a
dis
trib
utio
n of
rol
es a
nd r
espo
nsib
ilitie
s, a
dmin
istr
ativ
ede
tails
, and
bud
get.
5. S
tude
nt s
elec
tion.
Spe
cify
targ
et s
tude
nts
for
the
cour
se a
nd th
e nu
mbe
r of
stu
dent
s to
be
trai
ned
each
tim
e th
e co
urse
isof
fere
d. W
orki
ng w
ith o
ther
s, s
elec
t sui
tabl
e st
uden
ts f
or th
e in
trod
uctio
n.
6. L
ocat
ion
Wor
king
with
oth
er m
embe
rs o
f yo
ur te
am, i
dent
ify
a su
itabl
e si
te f
or th
e in
trod
uctio
n. I
f po
ssib
le, o
ffer
the
cour
seat
a c
lient
's lo
catio
n.
7. M
ater
ials
Mak
e m
ultip
le c
opie
s of
Stu
dent
Gui
des,
Ins
truc
tor
Gui
des,
and
oth
er c
ours
e m
ater
ials
. Pac
kage
thes
e m
ater
ials
and
ship
them
to th
e co
urse
loca
tion,
wel
l ahe
ad o
f th
e sc
hedu
led
date
.
8. I
nstr
ucto
r Id
entif
y re
quir
emen
ts f
or in
stru
ctor
s fo
r th
e co
urse
. Sel
ect a
sui
tabl
e in
stru
ctor
for
the
intr
oduc
tion.
Tra
in th
ein
stru
ctor
to im
plem
ent t
he c
ours
e
224
225
94
9. M
onito
ring
. Sup
ervi
se th
e ac
tiviti
es o
f th
e in
stru
ctor
. Pro
vide
nec
essa
ry g
uida
nce
and
feed
back
.
10. R
evis
ion.
Alth
ough
"fi
nal"
rev
isio
ns h
ave
been
com
plet
edea
rlie
r af
ter
the
fiel
d te
st, y
ou m
ay id
entif
y a
few
new
pro
blem
s
duri
ng th
e in
trod
uctio
n. M
ake
suita
ble
revi
sion
s as
soo
n as
pos
sibl
e.
227
226
95
How
to P
re a
re I
nstr
ucto
rs
1In
stru
ctor
rol
e. A
s yo
u de
velo
p th
e co
urse
, pay
spe
cial
atte
ntio
n to
the
role
s an
d re
spon
sibi
litie
s of
the
inst
ruct
or. T
hese
may
rang
e fr
om a
dmin
iste
ring
a s
elf-
inst
ruct
iona
l cou
rse
(in
whi
ch c
ase
the
inst
ruct
or w
ill b
e re
ferr
ed to
as
the
cour
sead
min
istr
ator
)to
prov
idin
g al
l the
con
tent
and
act
iviti
es. M
ake
a no
te o
f al
l maj
or in
stru
ctor
act
iviti
es.
2. G
uide
pre
para
tion.
Pre
pare
a c
ompr
ehen
sive
gui
de f
or a
n in
stru
ctor
-led
, mat
eria
ls-b
ased
cou
rse.
With
the
othe
r ty
pes
of c
ours
es,
prep
are
a su
itabl
e in
stru
ctor
or
adm
inis
trat
or g
uide
.
3. I
nstr
ucto
r re
quir
emen
ts. T
hrou
ghou
t the
dev
elop
men
t of
the
cour
se, n
ote
dow
n va
riou
s re
quir
emen
ts f
or th
e in
stru
ctor
. Cla
ssif
yth
ese
requ
irem
ents
in te
rms
of s
ubje
ct-m
atte
r ex
pert
ise,
trai
ning
ski
lls, a
nd im
plem
enta
tion
skill
s.
4. S
elec
tion.
Wor
king
with
the
clie
nt, T
rain
ing
Man
ager
, and
Tra
inin
g D
eliv
ery
pers
onne
l, de
cide
how
man
y in
stru
ctor
s ar
e ne
eded
to im
plem
ent t
he c
ours
e. I
dent
ify
and
recr
uit s
uita
ble
cand
idat
es f
or th
e in
stru
ctor
's jo
b.
5. F
amili
ariz
atio
n. W
alk
the
inst
ruct
ors
thro
ugh
the
cour
se m
ater
ials
. Hav
e th
e in
stru
ctor
rev
iew
the
mat
eria
ls a
nd p
repa
re a
list
of
ques
tions
. Cla
rify
the
cont
ent a
nd f
orm
at o
f th
e co
urse
.
6. T
rain
ing.
Tra
in th
e in
stru
ctor
in a
s cl
ose
to a
n on
-the
-job
situ
atio
n as
pos
sibl
e. H
ere
is a
n ap
prop
riat
e tr
aini
ng s
eque
nce:
Hav
e th
e in
stru
ctor
atte
nd a
s a
stud
ent w
hile
you
teac
h th
e co
urse
.
Hav
e th
e in
stru
ctor
obs
erve
you
whi
le y
ou te
ach
the
cour
t,H
ave
the
inst
ruct
or te
ach
som
e pa
rts
of th
e co
urse
whi
le y
ou te
ach
the
othe
rs.
Ask
the
inst
ruct
or to
teac
h th
e co
urse
whi
le y
ou o
bser
ve.
Ask
the
inst
ruct
or to
teac
h th
e co
urse
on
his
or h
er o
wn.
Be
avai
labl
e fo
r as
sist
ance
7. F
ollo
w-u
p M
onito
r th
e in
stru
ctor
's p
erfo
rman
ce f
rom
tim
e to
tim
e. P
rovi
de a
ppro
pria
tesu
gges
tions
to th
e in
stru
ctor
.
229
228
96
How
To
Prep
are
Inst
ruct
iona
l Mat
eria
ls
I. M
aste
r co
py. S
oon
afte
r th
e fi
eld
test
, mak
e fi
nal c
hang
es in
the
inst
ruct
iona
l pac
kage
. Pre
pare
a m
aste
r co
py o
fall
cour
se
mat
eria
ls.
2. L
ist.
Prep
are
a lis
t of
all m
ater
ials
for
the
cour
se. B
e su
re to
incl
ude
the
follo
win
g:Pr
ecou
rse
mat
eria
lsSt
uden
t Gui
deO
ther
stu
dent
use
mat
eria
lsJo
b ai
dsC
ase
mat
eria
lsIn
stru
ctor
Gui
deB
ackg
roun
d re
adin
gs f
or th
e in
stru
ctor
Ove
rhea
d tr
ansp
aren
cies
Med
iate
d m
ater
ials
(vi
deot
ape,
aud
iota
pe, o
r sl
ides
)T
ests
(an
d ot
her
mea
suri
ng in
stru
men
ts)
3. M
ock-
up. P
rodu
ce o
ne c
opy
of e
ach
mat
eria
l in
its f
inal
for
m. I
f th
e pa
ges
are
to b
e co
pied
fro
ntan
d ba
ck, v
erif
y th
ey a
reco
pied
cor
rect
ly. A
ssem
ble,
pun
ch, a
nd b
ind
the
mat
eria
ls in
app
ropr
iate
bin
ders
.
4. D
uplic
atio
n. D
ecid
e ho
w m
any
copi
es o
f ea
ch m
ater
ial a
re n
eede
d. B
e su
re e
ach
stud
ent w
ill h
ave
a co
pyof
rel
evan
t mat
eria
lsan
d a
few
add
ition
al c
opie
s w
ill b
e av
aila
ble
for
the
inst
ruct
or a
nd a
ny o
bser
vers
. Hav
e co
pies
dup
licat
ed,
colla
ted,
bou
nd, a
nd
pack
aged
.
5. S
hipm
ent.
Dec
ide
how
the
mat
eria
ls a
re to
be
dist
ribu
ted
to th
e st
uden
ts. M
ail o
ut p
reco
urse
mat
eria
ls w
ell a
head
of
time.
Shi
pco
pies
of
stud
ent m
ater
ials
and
oth
er m
ater
ials
to th
e co
urse
loca
tion,
wel
l ahe
ad o
f tim
e. A
s a
back
up,
hand
car
ry a
mas
ter
copy
of
all m
ater
ials
whe
n yo
u tr
avel
.
6. D
aily
che
ck. A
t the
beg
inni
ng o
f ea
ch d
ay o
f th
e co
urse
, pre
pare
a c
heck
list o
f m
ater
ials
requ
ired
for
the
day.
Dou
ble
chec
k th
at
thes
e m
ater
ials
are
ava
ilabl
e.
231
230
97
7. D
istr
ibut
ion.
At t
he b
egin
ning
of
the
firs
t day
of
the
cour
se, d
istr
ibut
e al
l stu
dent
mat
eria
ls. W
alk
the
stud
ents
thro
ugh
diff
eren
tm
ater
ials
and
ver
ify
they
hav
e al
l mat
eria
ls. Y
ou m
ay d
ecid
e to
with
hold
som
e m
ater
ials
(e.
g., s
choo
l sol
utio
ns)
to p
reve
nt th
est
uden
ts f
rom
inte
ntio
nally
or
inad
vert
ently
look
ing
ahea
d. H
owev
er, r
emem
ber
the
mor
e di
ffer
ent p
iece
s yo
u ha
ve, t
he g
reat
er th
eda
nger
of
mis
sing
mat
eria
ls.
8. F
ollo
w-u
p. A
t the
end
of
each
day
, col
lect
the
cour
se m
ater
ials
and
rep
lace
them
in th
e pa
ckag
e. F
or e
xam
ple,
col
lect
and
orga
nize
all
tran
spar
enci
es in
a b
inde
r so
that
they
are
rea
dy th
e ne
xt ti
me.
233
232
98
owre
are
acs
thes
an
Plan
ning
fac
tors
Lis
ting
requ
irem
ents
Roo
mar
rang
emen
tSp
ace
Med
iaeq
uipm
ent
Wor
k eq
uipm
ent
Ver
ific
atio
n
Cift
iCC
OM
CC
itt
'
234
BE
ST
CO
PY
AV
AIL
AB
LE23
5
How
To
Prep
are
Faci
litie
s an
d E
quip
men
t
1. P
lann
ing
fact
ors.
In
prep
arin
g fa
cilit
ies
and
equi
pmen
t for
the
cour
se. t
hink
of
the
num
ber
and
type
of
stud
ents
, the
dur
atio
n of
the
cour
se. a
nd th
e pr
imar
y de
liver
y st
rate
gy. T
hese
fac
tors
det
erm
ine
the
faci
litie
s an
d th
e eq
uipm
ent y
ou n
eed.
2. L
istin
g re
quir
emen
ts. W
alk
thro
ugh
each
less
on o
r m
odul
e of
the
cour
se a
nd m
ake
a lis
t of
equi
pmen
t and
fac
ilitie
s ne
eded
.Id
entif
y ty
pes
of a
ctiv
ities
(e.
g., i
ndiv
idua
l stu
dy, s
mal
l-gr
oup
role
-pla
y, o
r la
rge
grou
p pr
esen
tatio
n).
3. R
oom
arr
ange
men
t. Sp
ecif
y th
e fu
rnitu
re a
nd r
oom
arr
ange
men
t. H
ere
are
four
gen
eral
arr
ange
men
ts:
11T
heat
er s
tyle
with
cha
irs
in r
ows.
Thi
s is
app
ropr
iate
if th
e co
urse
pri
mar
ily r
evol
ves
arou
nd le
ctur
es a
nd m
edia
ted
pres
enta
tions
.C
lass
room
arr
ange
men
t with
tabl
e sp
ace
for
each
stu
dent
. Thi
s is
app
ropr
iate
if th
e st
uden
t has
to d
o a
lot o
f w
ritin
g.
H B
anqu
et a
rran
gem
ent w
ith s
mal
l (us
ually
rou
nd)
tabl
es w
ith c
hair
s ar
ound
them
. Thi
s ar
rang
emen
t is
appr
opri
ate
if th
eco
urse
invo
lves
sev
eral
sm
all-
grou
p ac
tiviti
es.
U s
hape
with
tabl
es a
nd c
hair
s in
a s
emic
ircl
e an
d th
e in
stru
ctor
in f
ront
. Thi
s ar
rang
emen
t is
appr
opri
ate
for
cour
ses
with
few
er th
an 1
5 st
uden
ts.
4. S
pace
. Spe
cify
how
muc
h sp
ace
is n
eede
d in
the
clas
sroo
m. I
f yo
u ar
e pl
anni
ng to
cha
nge
the
arra
ngem
ent f
requ
ently
, mak
e su
reth
ere
is e
noug
h ro
om to
mov
e th
e ch
airs
and
tabl
es a
roun
d.
5 M
edia
equ
ipm
ent.
Mak
e a
list o
f m
edia
equ
ipm
ent n
eede
d fo
r ea
ch le
sson
. Be
spec
ific
for
exa
, 1pl
e, if
you
are
goi
ng to
use
vide
otap
es, s
peci
fy th
e fo
rmat
Inc
lude
TV
mon
itors
and
scr
eens
(fo
r ov
erhe
ad p
roje
ctio
n) in
the
list.
6. W
ork
equi
pmen
t. Y
our
cour
se m
ay in
volv
e de
mon
stra
tions
and
han
ds-o
n pr
actic
e on
dif
fere
nt e
quip
men
t use
don
-the
-job
(e.
g.,
a pe
rson
al c
ompu
ter)
. Rev
iew
the
Inst
ruct
or G
uide
and
oth
er c
ours
edo
cum
ents
to id
entif
y w
hat k
inds
of
equi
pmen
t are
nee
ded
whe
n.
7 V
erif
icat
ion.
Upo
n ar
riva
l at t
he c
ours
e lo
catio
n, c
heck
all
of th
e eq
uipm
ent (
and
the
righ
t typ
e of
equ
ipm
ent)
are
avai
labl
e. I
f
your
cou
rse
is o
f m
ore
than
a d
ay's
dur
atio
n, c
heck
the
equi
pmen
t at t
he b
egin
ning
of
each
day
.
236
237
99
Dat
a C
olle
ctio
n in
Fol
low
-Up
Eva
luat
ion
To
answ
er th
ese
follo
w-u
p qu
estio
nsU
se th
ese
data
sou
rces
and
str
ateg
ies:
1. A
dequ
acy
of C
ours
e D
evel
opm
ent
Con
tent
ana
lysi
s of
:
Is th
e co
urse
des
igne
d to
sol
ve a
spe
cifi
c pe
rfor
man
ce p
robl
em?
Perf
orm
ance
Ana
lysi
s R
epor
t Lea
rner
Ana
lysi
s
Is th
e co
urse
bas
ed o
n th
e ch
arac
teri
stic
s an
d pr
efer
ence
s of
the
stud
ents
?T
rain
ing
Nee
ds A
naly
sis
Rep
ort T
ask
Det
ails
Wor
kshe
et
Is th
e co
nten
t of
the
cour
se b
ased
on
a sy
stem
atic
trai
ning
nee
ds a
naly
sis?
Skill
-Kno
wle
dge
Req
uire
men
ts W
orks
heet
Inst
ruct
iona
l Obj
ectiv
es W
orks
heet
Is th
e de
liver
y st
rate
gy f
or th
e co
urse
bas
ed o
n th
e tr
aini
ng n
eeds
ana
lysi
s da
ta?
Tes
ts (
and
Tes
t Spe
cifi
catio
ns)
Eva
luat
ion
Rep
orts
Are
the
skill
s an
d kn
owle
dge
taug
ht in
the
cour
se r
elev
ant t
o th
e jo
b?Fi
eld
Tes
t rep
ort F
inal
ver
sion
s of
all
cour
sem
ater
ial
Are
the
inst
ruct
iona
l obj
ectiv
es o
f th
e co
urse
cle
arly
spe
cifi
ed?
Inte
rvie
ws
with
Cou
rse
deve
lopm
ent t
eam
mem
bers
Are
the
test
s va
lid a
nd c
rite
rion
ref
eren
ced?
Fiel
d te
st s
tude
nts
Are
the
lear
ning
act
iviti
es e
ffec
tive?
Inst
ruct
ors
Is th
e co
urse
rev
ised
on
the
basi
s of
eva
luat
ion?
Do
the
fiel
d te
st d
ata
indi
cate
eff
ectiv
enes
s?
239
238
100
2. R
elia
bilit
y of
cou
rse
impl
emen
tatio
n
How
is th
e co
urse
sup
pose
d to
be
impl
emen
ted?
Is th
e im
plem
enta
tion
plan
pra
ctic
al?
Is th
ere
a di
scre
panc
y be
twee
n th
e di
rect
ives
and
the
actu
al im
plem
enta
tion
of th
e co
urse
?
Inte
rvie
ws
with
Ins
truc
tors
Obs
erva
tion
in s
ites
whe
re th
e co
urse
is b
eing
con
duct
ed
Stud
ents
, Cou
rse
Man
ager
s, I
nstr
ucto
rs
Wha
t are
the
caus
es o
f th
ese
disc
repa
ncie
s?C
onte
nt a
naly
sis
How
can
we
mak
e th
e co
urse
mor
e fe
asib
le?
Inst
ruct
or G
uide
, Adm
inis
trat
or G
uide
3. S
tude
nt r
eact
ions
Inte
rvie
ws
with
Wha
t are
the
posi
tive
reac
tions
of
the
stud
ents
to d
iffe
rent
asp
ects
of
the
New
stu
dent
s, G
radu
ates
cour
se?
Que
stio
nnai
re a
bout
the
cour
se a
dmin
iste
red
toW
hat a
re th
e fr
eque
nt s
tude
nt c
ompl
aint
s an
d co
ncer
ns?
Stud
ents
, Ins
truc
tors
Inte
rvie
ws
on s
tude
nt r
eact
ions
to th
e. c
ours
esw
ith I
nstr
ucto
rs, M
anag
ers
4. I
nstr
uctio
nal e
ffec
tiven
ess
Con
tent
ana
lysi
s of
Wha
t are
the
inst
ruct
iona
l gai
ns d
ue to
this
cou
rse?
How
eff
ectiv
ely
and
relia
bly
do th
e st
uden
ts a
chie
ve th
e co
urse
obj
ectiv
es?
240
Post
test
res
ults
Em
bedd
ed te
st r
esul
ts
Inte
rvie
ws
with
Stu
dent
s, I
nstr
ucto
rs
241
101
5. M
anag
emen
t rea
ctio
n
Wha
t are
the
posi
tive
man
agem
ent r
eact
ions
tow
ard
the
cour
se?
Man
ager
s, S
uper
viso
rs
Wha
t are
the
maj
or c
ompl
aint
s an
d co
ncer
ns o
f m
anag
ers
abou
t the
cou
rse?
Que
stio
nnai
res
for
Man
ager
s
Wha
t are
the
futu
re p
lans
for
the
cour
se?
Supe
r is
ors
6. R
elev
ance
Inte
rvie
ws
abou
t rel
evan
ce o
f th
e co
urse
with
How
rel
evan
t are
the
skill
s an
d kn
owle
dge
taug
ht in
this
cou
rse
to th
eG
radu
ates
, Sup
ervi
sors
, Man
ager
s, S
ubje
ct-
real
ities
of
the
wor
kpla
ce?
mat
ter
expe
rts
Wha
t are
the
prob
lem
s in
tran
sfer
ring
and
util
izin
g th
e sk
ills
from
the
cour
se?
Has
the
wor
k si
tuat
ion
or p
roce
dure
bee
n ch
ange
d? H
as th
e te
chno
logy
cha
nged
?
243
242
102
Form
ativ
e C
rite
ria
Impl
emen
tatio
n E
valu
atio
n
Que
stio
ns
Impl
emen
tatio
n pl
anni
ng
Prep
arat
ion
for
impl
emen
tatio
n?
Cou
rse
intr
oduc
tion
Follo
w-u
p ev
alua
tion
Follo
w-u
p im
pact 24
4
Was
a d
etai
led
and
real
istic
pla
n fo
rth
e im
plem
enta
tion
of th
e co
urse
prep
ared
?
Wer
e m
ater
ials
, ins
truc
tors
, and
faci
litie
s an
d eq
uipm
ent f
orim
plem
entin
g th
e co
urse
pre
pare
d in
a tim
ely
and
effe
ctiv
e fa
shio
n?
Was
the
cour
se in
trod
uced
in a
syst
emat
ic a
nd e
ffec
tive
man
ner?
Was
the
follo
w-u
p ev
alua
tion
plan
ned
syst
emat
ical
ly a
ndim
plem
ente
d ef
fect
ivel
y?
Doe
s th
e co
urse
res
ult i
n an
iden
tifia
ble
and
docu
men
ted
bene
fit
to th
e C
ompa
ny?
If n
ot, h
ave
suita
ble
reco
mm
enda
tions
mad
e to
revi
se it
?
Rat
ings
and
Com
men
tsL
OW
12
34
HIG
H5
245
103
Cou
rse
Mai
nten
ance
Che
cklis
t
I. T
he le
ngth
of
the
cour
se is
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: D
o al
l stu
dent
s fi
nish
the
cour
se?
Are
som
e un
its to
olo
ng o
r to
o sh
ort?
Doe
s th
e co
urse
pac
e se
em to
o fa
st o
r
too
slow
?
Com
men
ts:
2. T
he in
stru
ctor
gui
de is
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: I
s th
e te
chni
cal c
onte
nt c
ompl
ete?
Is
the
docu
men
tatio
nw
ell o
rgan
ized
? Is
the
form
at e
asy
to f
ollo
w?
Are
the
dire
ctio
ns c
lear
?
Com
men
ts:
3. T
he s
tude
nt m
ater
ials
are
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: A
re s
tude
nt m
ater
ials
com
plet
e? D
o th
ey lo
okpr
ofes
sion
ally
pro
duce
d? A
re th
ey
easy
to r
ead?
Do
stud
ents
ref
er to
them
?
Com
men
ts:
4. T
he m
edia
ted
mat
eria
ls u
sed
in th
e co
urse
are
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: A
re th
e m
edia
ted
mat
eria
ls s
uffi
cien
t? A
re s
lides
and
tran
spar
enci
es c
lear
? D
oes
the
med
ia e
nhan
ce th
e
lear
ning
? A
re th
ey a
ppro
pria
te?
Com
men
ts:
247
24(
104
5. T
he c
ours
e ob
ject
ives
are
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: A
, e, o
bjec
tives
cle
ar a
nd c
onci
se?
Do
they
str
ess
key
skill
s an
d kn
owle
dge?
Do
they
hel
p or
gani
ze th
em
ater
ials
?
Com
men
ts:
6. T
he te
sts
incl
uded
in th
e co
urse
are
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: A
re th
ese
test
s av
aila
ble?
Are
they
suf
fici
ent?
Are
sco
ring
gui
des
prov
ided
for
eac
h te
st?
Do
the
test
s m
easu
re r
equi
red
skill
s an
d kn
owle
dge?
Com
men
ts:
7. T
he e
xerc
ises
in th
e co
urse
are
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: A
re th
e ex
erci
ses
suff
icie
nt?
Are
they
cha
lleng
ing
and
mea
ning
ful?
Can
they
be
eval
uate
d?
Com
men
ts:
8. T
he te
chni
cal c
onte
nt o
f th
e co
urse
rep
rese
nts
the
curr
ent s
tate
-of-
the-
art:
adeq
uate
inad
equa
te5
43
21
249
248
105
Eva
luat
ion
Cri
teri
a: H
ow lo
ng h
as th
e co
urse
bee
n ta
ught
? H
ow r
ecen
tly h
as it
bee
nre
vise
d?
Com
men
ts:
9. T
he te
achi
ng s
trat
egie
s us
ed in
this
cou
rse
are:
adeq
uate
inad
equa
te5
43
21
Eva
luat
ion
Cri
teri
a: D
oes
the
cour
se p
rovi
de f
or s
tude
nt e
xerc
ise
and
eval
uatio
n? A
re s
tude
nts
invo
lved
? D
oes
lear
ning
take
pla
ce?
Com
men
ts:
10. T
he o
rgan
izat
ion
of th
e co
urse
is:
adeq
uate
inad
equa
te5
43
21
Eva
luat
ion
Cri
teri
a: D
oes
the
cour
se f
low
sm
ooth
ly?
Are
less
ons
and
units
inte
grat
ed?
Com
men
ts:
1 I
.T
he c
ase
prob
lem
use
d in
this
cou
rse
is:
adeq
uate
inad
equa
te5
43
21
Eva
luat
ion
Cri
teri
a: A
re th
e nu
mbe
r of
cas
e pr
oble
ms
adeq
uate
?D
o th
ey r
elat
e to
the
stud
ents
wor
k
expe
rien
ce?
Com
men
ts:
251
250
106
12. S
tude
nt in
tere
st a
nd m
otiv
atio
n in
the
cour
se is
.ad
equa
tein
adeq
uate
54
2
Eva
luat
ion
Cri
teri
a: D
o st
uden
ts r
eact
pos
itive
ly to
the
cour
se?
Do
stud
ents
see
m e
nthu
sias
tic in
cla
ss?
Com
men
ts:
13. S
tude
nt f
eedb
ack
on th
e co
urse
indi
cate
s th
e co
urse
is:
adeq
uate
inad
equa
te5
43
21
Eva
luat
ion
Cri
teri
a: I
s th
e st
uden
t fee
dbac
k su
mm
ary
data
pos
itive
? Is
the
stud
ent c
omm
ents
pos
itive
?
Com
men
ts:
14. T
he v
alue
of
the
cour
se to
the
stud
ents
job
need
s is
:ad
equa
tein
adeq
uate
54
32
1
Eva
luat
ion
Cri
teri
a: D
o st
uden
ts n
eed
this
cou
rse
to d
o th
eir
jobs
? D
oes
the
cour
se p
rovi
de th
em w
ith th
e re
quir
ed s
kills
and
know
ledg
e to
do
thei
r jo
bs?
Com
men
ts:
252
253
107
Sum
mat
ive
Eva
luat
ion
of T
rain
ing
254
255
EV
AL
UA
TIO
N O
F T
RA
ININ
G
The
inte
nt o
f th
is s
ectio
n is
to p
rovi
de a
fra
mew
ork
for
eval
uatio
n an
d re
visi
on v
ia th
e pe
rson
al c
ompu
ter
i.e. a
sor
tof
job
aid
that
whe
n im
plem
ente
d sh
ould
org
aniz
e an
dsi
mpl
ify
the
proc
ess.
I w
ill d
eal w
ith s
umm
ativ
e ev
alua
tion
only
in th
is p
aper
,
alth
ough
this
is o
nly
part
of
the
eval
uatio
n st
rate
gy.
Form
ativ
e ev
alua
tion
the
othe
r ha
lf o
f a
Eva
luat
ion
syst
em o
f co
urse
,is
the
subj
ect o
f th
is e
ntir
e bo
ok.
The
mos
t use
ful m
odel
of
eval
uatio
n I
have
fou
nd is
Kir
kpat
rick
s le
vel o
f ev
alua
tion
or th
e ad
apta
tion
byB
reth
ower
and
Rum
ler
.T
he to
ol c
an b
e us
ed a
s fo
llow
s: L
evel
1 A
re th
etr
aine
es h
appy
? L
evel
2 D
oes
the
trai
ning
do
wha
t it i
ssu
ppos
ed to
do?
Lev
el 3
Are
the
conc
epts
use
d (
see
atta
ched
sur
vey
pg.
SE 8
-10)
Lev
el 4
Doe
s ap
plic
atio
n of
the
conc
epts
affe
ct th
e
orga
niza
tion?
For
eac
h of
thes
e le
vels
the
who
- so
urce
of
data
,w
hat-
to m
easu
re, w
here
- fi
nd d
ata,
why
- cr
iteri
a, a
ndho
w-
data
gath
erin
g m
etho
d m
ust b
e an
swer
ed.
The
rol
e of
the
trai
ning
fun
ctio
n w
ith r
espe
ct to
follo
w u
p is
to a
sses
s th
e ex
tent
the
new
idea
s, s
kills
and
mot
ivat
ion
will
get a
nat
ural
opp
ortu
nity
for
exe
rcis
ean
d re
info
rcem
ent.
And
hav
ing
done
this
tore
com
men
d st
eps
to o
verc
ome
any
obst
acle
s.
256
257
108
Thi
s te
nds
to b
e ov
erlo
oked
by
the
Tra
inin
g M
anag
er w
ho o
ften
om
its to
con
duct
any
fol
low
up.
We
tend
to lo
ok a
t the
com
plet
ion
of a
pro
gram
as
the
end
of r
espo
nsib
ility
. Thi
s at
titud
e in
dica
tes
insu
ffic
ient
und
erst
andi
ng o
f th
e le
arni
ng p
roce
ss.
One
has
to b
e th
ankf
ul th
at th
e na
tura
l inc
linat
ions
of
peop
le to
see
k re
info
rcem
ent o
n th
eir
own
with
out w
aitin
g fo
r in
tere
st to
be
show
n by
oth
ers
.
258
259 10
9
Plan
ning
Eva
luat
ion
of tr
aini
ng b
egin
s lik
e ev
ery
othe
r bu
sine
ss f
unct
ion
with
Pla
nnin
g. T
he c
ore
com
pete
ncie
s of
the
ente
rpri
se m
ust
firs
t be
dete
rmin
ed a
nd s
tate
d th
is b
ecom
es th
e in
put t
o th
e tr
aini
ng a
nd e
valu
atio
n su
bsys
tem
s. T
he tr
aini
ng s
trat
egy
is n
ext
deve
lope
d be
ginn
ing
with
a m
issi
on s
tate
men
t whi
ch in
clud
es th
is s
trat
egy
: the
trai
ning
str
ateg
y is
to tr
ain
to h
elp
deve
lop,
upd
ate,
and
mai
ntai
n co
re c
ompe
tenc
y es
tabl
ishe
d by
the
busi
ness
Car
r (J
une
92' T
rain
ing)
The
trai
ning
sys
tem
s pu
rpos
e is
to e
nabl
e so
meo
ne to
impr
ove
thei
r pe
rfor
man
ce C
arr
(Jun
e 92
' Tra
inin
g). T
his
is
dist
ingu
ishe
d fr
om E
duca
tion
and
Dev
elop
men
t whi
ch s
houl
d be
mea
sure
d in
a w
hole
dif
fere
nt m
anne
r du
e to
late
ntle
arni
ng a
nd
the
esta
blis
hmen
t of
lifel
ong
patte
rns
of le
arni
ng a
nd k
now
ledg
e ac
quir
ed.
Eva
luat
ion
shou
ld th
en b
e lo
oked
at a
s pa
rt o
f a
broa
der
perf
orm
ance
sys
tem
i.e.
a s
ubsy
stem
with
in tr
aini
ng w
hich
in it
self
is a
sub
syst
em o
f th
e la
rger
bus
sys
tem
. One
of
the
goal
s of
trai
ning
is to
incr
ease
the
utili
ty o
f tr
aini
ngde
fine
d as
:
UT
ILIT
Y =
EFF
EC
TIV
EN
ESS
+ E
FFIC
IEN
CY
i.e.
you
can
add
tim
e to
a c
ours
e w
hich
will
low
er th
e ef
fici
ency
but w
ill
incr
ease
the
lear
ning
(ef
fect
iven
ess)
. Sin
ce th
is,:s
sent
ially
a tr
ade-
off
proc
ess
( on
e in
crea
ses
whi
le th
e ot
her
decr
ease
s) th
e on
ly
way
to in
crea
se u
tility
is b
rpro
cess
impr
ovem
ent C
arr
(Jun
e 92
' Tra
inin
g). T
rain
ing
look
ed a
t as
a sy
stem
has
an
Inpu
t, a
Proc
ess,
and
an O
utpu
t Fee
dbac
k in
the
form
of
thes
e se
vera
l tec
hniq
ues
men
tione
d be
low
and
acc
ompa
ny r
evis
ion
are
para
mou
nt .
The
eval
uatio
n sy
stem
with
in tr
aini
ng th
en s
houl
d be
bas
ed o
n th
e th
is u
tility
for
mul
a.
261
260
110
Philo
soph
y of
Eva
luat
ion
Dug
an L
aird
(19
78 )
sta
tes
Tra
inin
g ev
alua
tion
is b
ased
on
thre
e ph
iloso
phy
all o
f w
hich
are
impo
rtan
t but
one
of
whi
ch is
pri
mar
y
(the
one
util
ized
whe
n pu
sh c
omes
to s
hove
) th
ey a
re:
1.C
ontr
ibut
ion
to g
oal b
otto
m li
ne r
esul
ts.
2.A
chie
vem
ent o
f le
arni
ng o
bjec
tive.
( r
ight
obj
ectiv
e ta
ught
and
lear
ned,
so
you
incr
ease
the
likel
ihoo
d th
ey w
ill u
se, b
ack
on jo
b
i.e. a
noth
er d
epar
Tea
cher
/Ins
truc
tore
nts
prob
lem
.
3.Pe
rcep
tion
of w
orth
. (se
t env
iron
men
t for
lear
ning
let l
earn
er ju
dge
utili
ty, w
orth
and
val
ue)
Goo
d tr
aini
ng lo
oks
at a
ll th
ree
and
wha
t you
rep
ort a
nd w
hat y
ou d
o no
t ref
lect
s w
hat y
ou th
ink
is im
port
ant.
Lev
els
of E
valu
atio
n
Lev
el O
ne Mos
t tra
inin
g is
mea
sure
d at
this
leve
l and
som
e at
leve
l tw
o. (
Gor
don
Tra
inin
g A
ugus
t 199
1). W
e m
ake
a m
ista
ke h
ere
by
wor
ryin
g ab
out t
he tr
aini
ng f
unct
ion
as s
epar
ate
from
the
larg
er p
erfo
rman
ce s
yste
m a
nd c
once
ntra
te o
n di
d yo
u lik
e it
or d
id y
ou
feel
the
trai
ning
was
goo
d.
"Hap
py s
heet
's",
are
they
are
ref
erre
d to
are
oft
en s
neer
ed a
t by
thos
e th
at s
ay it
doe
s no
t mea
sure
qual
ity b
ut th
e en
tert
ainm
ent v
alue
of
trai
ning
.
269
263
111
Thi
s ne
ed n
ot b
e th
e ca
se, l
evel
one
can
tell
you
impo
rtan
t inf
orm
atio
n ab
out I
nitia
l cus
tom
ersa
tisfa
ctio
n bu
t don
't pr
eten
d
it is
telli
ng y
ou s
omet
hing
abo
ut h
ighe
r le
vels
of
eval
uatio
n. (
Gor
don
1991
.). T
he m
ost
impo
rtan
t use
is to
tell
if a
pre
sent
atio
n
tech
niqu
e is
add
ing
or d
etra
ctin
g fr
om th
e le
arni
ng p
roce
ss i.
e. th
e ac
hiev
emen
t of
obje
ctiv
es.
Cog
nitiv
e Ps
ycho
logy
str
esse
s th
e ac
quis
ition
of
mat
eria
l and
sch
ema
deve
lopm
ent
the
pres
entin
g of
mat
eria
l sho
uld
be s
uch
that
it e
nhan
ces
thes
e pr
oces
ses.
this
is a
ccom
plis
hed
by o
ne o
r m
ore
of th
e fo
llow
ing
met
hods
whe
n ap
plic
able
:
I.or
gani
zatio
n of
mat
eria
l
2fr
eque
nt r
evie
w
3.re
leva
nt e
xam
ples
4.pr
actic
e an
d fe
edba
ck o
n pe
rfor
man
ce
5.en
gagi
ng th
e le
arne
r (i
nvol
vem
ent)
Wha
t is
actu
ally
nee
ded
(and
cou
ld b
e en
hanc
ed b
y th
e us
e of
spr
eads
heet
s)i
s a
stat
istic
ally
val
id s
tudy
of
the
corr
elat
ion
betw
een
tech
niqu
es g
athe
red
from
the
leve
l one
que
stio
nnai
re a
nd th
e le
arni
ngof
the
obje
ctiv
es
Lev
el T
wo
A m
easu
re o
f th
e le
arni
ng th
at o
ccur
red
(if
any)
. Whi
le th
ere
are
man
ym
etho
ds f
rom
pap
er a
nd p
enci
l tes
ts to
act
ual
perf
orm
ance
of
the
task
whe
reve
r po
ssib
le th
e cl
oser
test
ing
is to
the
real
back
on
the
job
perf
orm
ance
nee
ded
the
grea
ter
the
264
265
112
chan
ce o
f tr
ansf
er. A
t thi
s le
vel w
e ne
ed to
be
cert
ain
we
reco
gniz
e an
d di
stin
ctio
n be
twee
n kn
owle
dge
and
appl
icat
ion
of th
e
skill
. Pas
sing
a p
aper
and
pen
cil t
est o
n th
e ha
ndlin
g of
Sul
furi
c ac
id d
oes
not a
ssur
e a
cata
stro
phe
will
not
res
ult t
he f
irst
tim
e a
pers
on e
ncou
nter
s th
e re
al th
ing.
It i
s no
t a m
atte
r of
sim
ply
did
they
lear
n w
hat y
ou w
ante
d th
em to
, but
rat
her
is it
wha
tyou
rea
lly
wan
ted
to a
ccom
plis
h? T
here
is n
o si
mpl
e re
gurg
itatio
n of
info
rmat
ion
that
will
sub
stitu
te f
or a
ctua
l dem
onst
ratio
n of
the
perf
orm
ance
this
is n
ot E
duca
tion
i.e. t
he s
choo
l sys
tem
of
our
child
hood
.
Lev
el 3
Tra
nsfe
r of
trai
ning
bac
k to
the
job
( se
e T
rain
ing
Follo
w-U
p Su
rvey
) m
eans
the
exte
nt to
whi
ch th
e sk
ills
lear
ned
in a
trai
ning
env
iron
men
t are
act
ual a
pplie
d to
the
job.
It i
s ge
nera
lly r
ecom
men
ded
that
this
eva
luat
ion
be c
ondu
cted
3 to
6 m
onth
s
afte
r co
mpl
etio
n of
trai
ning
by
a or
gani
zatio
n in
depe
nden
t of
the
Tra
inin
g O
rgan
izat
ion.
The
foc
us o
f th
e ev
alua
tion
is r
eally
to
dete
rmin
e w
hy o
r w
hy n
ot th
e tr
aini
ng is
bei
ng u
sed
.Rec
allin
g ou
r go
al o
f m
axim
izin
g ut
ility
, it i
s eq
ually
impo
rtan
t thi
s an
alys
is
shou
ld b
e sc
rutin
ized
for
oth
er m
ore
effi
cien
t met
hods
of
achi
evin
g ou
r ob
ject
ives
i.e.
job
aids
and
CB
T (
part
icul
arly
impo
rtan
t if
just
in ti
me
trai
ning
is im
port
ant t
o ou
r bu
sine
ss.)
Succ
ess
in th
is p
hase
like
the
othe
r in
volv
es p
lann
ing
thro
ugho
ut th
e tr
aini
ng c
ycle
as
follo
ws:
1.In
itial
ly, t
he f
amili
ariz
atio
n an
d th
e su
ppor
t of
supe
rvis
ion
for
the
ende
avor
.
2.T
he o
peni
ng o
f th
e co
urse
sta
ting
goal
s an
d ex
pect
atio
ns o
f th
e co
urse
incl
udin
g ho
w it
will
be
used
bac
k on
the
job.
.
3.L
earn
er c
ontr
acts
whi
ch in
clud
e th
e in
tend
ed a
pplic
atio
ns o
f th
e le
arni
ng
267
266
113
4.Po
st c
ours
e su
ppor
t pro
vide
d by
one
the
follo
win
g:
Post
trai
ning
dis
cuss
ions
trai
nee/
supe
rvis
or
Supp
ort g
roup
s
Lev
el 4
Eff
ects
on
botto
m li
ne b
usin
ess
resu
lts (
pro
fit,
sale
s, q
uant
ity, q
ualit
y, s
afet
y et
c.)
If w
e fo
cuse
d on
our
Tra
inin
g St
rate
gy
(def
ined
abo
ve)
and
perf
orm
ance
and
trai
ning
nee
d an
alys
is a
re c
orre
ctly
don
e th
is b
ecom
e a
easy
mat
ter
of tr
aini
ng to
cur
e a
perf
orm
ance
dis
crep
ancy
. Per
form
ance
Ana
lysi
s sh
ould
hav
e di
scov
ered
wha
t the
pro
blem
ig c
ostin
g.an
d ev
alua
tion
to d
eter
min
e
to w
hat e
xten
t tra
inin
g cu
re it
aft
er th
e fa
ct s
houl
d pr
oduc
e so
me
reas
onab
le r
esul
ts. D
ana
Rob
inso
n (
1991
) st
ates
the
key
to g
ood
eval
uatio
n an
d go
od tr
aini
ng is
to e
stab
lishe
d a
caus
al li
nk b
etw
een
a sk
ill d
efic
ienc
y an
d a
busi
ness
pro
blem
pre
fera
bly
one
that
can
easi
ly b
e qu
antif
ied
atta
ched
. Bus
ines
s re
sults
are
then
a s
impl
e m
atte
r of
trac
king
the
sam
e in
dica
tors
you
trac
ked
befo
re th
e
prog
ram
beg
an; n
ow y
ou w
ant t
o se
e if
any
thin
g ha
ppen
ed. M
ost o
f us
get
into
trou
ble
by p
oor
plan
ning
and
slip
snod
exe
cutio
n of
the
anal
yses
pha
se th
e bu
sine
ss r
esul
ts a
re v
ery
diff
icul
t to
asce
rtai
n af
ter
the
fact
. It i
s to
fis
hing
for
res
ults
i.e.
we
did
not k
now
wha
t we
wer
e do
ing
now
we
wan
t to
see
if a
nyth
ing
happ
ened
.
269
268
114
Sum
mar
y
Tra
inin
g is
par
t of
a la
rger
sys
ten.
whi
ch h
as g
oals
it w
ishe
s to
ach
ieve
. Tra
inin
g st
rate
gy n
eed
tobe
link
ed to
thes
e go
als.
Tra
inin
g th
en m
ust b
e vi
ewed
as
a pr
oces
s w
ith in
puts
pro
cess
es a
nd o
utpu
ts .
Util
ity m
easu
res
the
effe
ctiv
enes
s of
the
trai
ning
syst
em. T
o in
crea
se u
tility
then
is to
impr
ove
the
proc
ess.
Met
rics
on
trai
ning
sho
uld
thus
pro
vide
fee
dbac
k on
how
trai
ning
cou
ld
be im
prov
ed i.
e. h
ow to
incr
ease
util
ity b
y ef
fect
iven
ess
or e
ffic
ient
. T
his
proc
ess
of f
eedb
ack
and
subs
eque
nt r
evis
ion
is
para
mou
nt to
any
sys
tem
. Spr
eads
heet
s pr
ovid
e a
mea
nsfo
r da
ta c
olle
ctio
n an
d an
alys
es w
hich
sim
plif
y th
e pr
oces
s.
2'/O
271
115
TR
AIN
ING
FO
LL
OW
-UP
SUR
VE
Y
Cou
rse
Titl
eD
ates
Atte
nded
Cou
rse
Obj
ectiv
e
A. A
t the
end
of
the
cour
se, t
o w
hat d
egre
e di
d yo
u fe
el th
at y
ou a
chie
ved
the
obje
ctiv
e st
ated
LIT
TL
E1
2SO
ME
3
VE
RY
MU
CH
45
IF Y
OU
CIR
CL
ED
less
TH
AN
A 3
ON
QU
EST
ION
A. S
TO
P H
ER
E A
ND
RE
TU
RN
TH
E Q
UE
STIO
NN
AIR
E,
OT
HE
RW
ISE
CO
NT
INU
E.
B.
Sinc
e co
mpl
etin
g th
e co
urse
, how
oft
en h
ave
you
used
the
skill
s th
at y
ou le
arne
d in
the
clas
s on
you
r jo
b as
sign
men
ts?
NE
VE
R
1
OC
CA
SIO
NA
LL
YD
AIL
Y
23
45
C.
As
a re
sult
of th
is c
ours
e, h
ow m
uch
the
trai
ning
hel
ped
you
in c
ompl
etin
g yo
ur jo
b as
sign
men
ts?
LIT
TL
E/N
OSO
ME
MA
JOR
IMPR
OV
EM
EN
TIM
PRO
VE
ME
NT
IMPR
OV
EM
EN
T1
23
45
IF Y
OU
AN
SWE
RE
D Q
UE
STIO
NS
B O
R C
WIT
H A
3 O
R G
RE
AT
ER
, TH
EN
GO
ON
TO
QU
EST
ION
E. I
FY
OU
AN
SWE
RE
D Q
UE
STIO
NS
B O
R C
WIT
H L
ESS
TH
AN
A 3
, TH
EN
GO
ON
TO
QU
EST
ION
F.
273
272
116
E. P
lace
a c
heck
nex
t to
each
rea
son
belo
w th
at m
ight
exp
lain
why
you
hav
e ap
plie
d th
e sk
ills
you
lear
ned
toyo
ur jo
b as
sign
men
ts:
My
supe
rvis
or r
equi
red
me
to u
se th
e ne
w s
kills
.I
rece
ived
hel
p fr
om o
ther
s in
my
wor
k ar
ea.
I w
as g
iven
nec
essa
ry ti
me
and/
or to
ols
to a
pply
the
skill
s.I
rece
ived
trai
ning
at t
he r
ight
tim
e to
pro
vide
me
with
the
skill
s w
hen
1 ne
eded
.T
he s
kills
I le
arne
d ap
plie
d di
rect
ly to
my
job
assi
gnm
ent.
My
supe
rvis
or d
iscu
ssed
with
me
how
my
new
ski
lls w
ould
1:1
' use
d on
my
job.
Oth
er p
leas
e lis
t oth
er f
acto
rs th
at h
elpe
d yo
u ap
ply
thes
e sk
ills
to y
our
job.
274
275
117
F. P
lace
a c
heck
nex
t to
each
rea
son
belo
w th
at c
ould
exp
lain
why
you
hav
e no
t bee
n su
cces
sful
in a
pply
ing
skill
s le
arne
d to
you
r
job
assi
gnm
ents
: My
supe
rvis
or d
id n
ot r
equi
re m
e to
use
the
skill
s.M
y su
perv
isor
was
not
aw
are
of w
hat s
kills
I le
arne
d.I
was
not
giv
en ti
me/
tool
s to
impl
emen
t the
ski
lls o
n th
e jo
b.T
here
was
no
one
to h
elp
me
impl
emen
t the
ski
lls in
my
wor
k ar
ea.
The
ski
lls d
id n
ot s
eem
to a
pply
to m
y jo
b as
sign
men
t.M
y jo
b as
sign
men
t cha
nged
so
thes
e sk
ills
did
not a
pply
.T
he tr
aini
ng w
as n
ot ti
med
rig
ht f
or m
y jo
b as
sign
men
t.O
ther
. Ple
ase
desc
ribe
oth
er r
easo
ns y
ou d
id n
ot a
pply
the
skill
s to
you
r jo
b
assi
gnm
ents
.
276
277
118
Sele
cted
Bib
liogr
aphy
Abe
lla, K
ay B
uild
ing
Succ
essf
ul T
rain
ing
Prog
ram
s. A
ddis
on-W
esle
y 19
86.
Bre
thow
er a
nd R
um le
r Im
prov
ing
Hum
an P
erfo
rman
ce Q
uart
erly
. 197
6
Car
r, C
lay,
"T
he T
hree
R's
of
Tra
inin
g,"
TR
AIN
ING
Mag
azin
e,pp
.60-
67, J
une
92.
Gilb
ert,
T. H
uman
Com
pete
nce:
Eng
inee
ring
Wor
thy
Perf
orm
ance
. New
Yor
k: M
cGra
w-H
ill, 1
978.
Gor
doi J
., "M
easu
ring
the
"Goo
dnes
s'of
Tra
inin
g,"
TR
/M
agaz
ine,
pp.
19-2
5, A
ugus
t 91.
Kir
kpat
rick
. D.L
. Eva
luat
ing
Tra
inin
g Pr
ogra
ms,
AST
D. 1
975.
Lai
rd,D
ugan
, "W
hy H
ave
a T
rain
ing
Dep
artm
ent A
nyw
ay,"
TR
AIN
ING
Mag
azin
e,pp
.16-
20, O
ctob
er 7
9.
Mag
er, R
ober
t, F.
, Wha
t Eve
ry M
anag
er S
houl
d K
now
Abo
utT
rain
ing.
Lak
e Pu
blis
hing
, Bel
mon
t, C
alif
. 199
2 C
h 1.
The
Nam
e (
Nils
on, C
arol
yn, T
rain
ing
Prog
ram
Wor
kboo
k an
d K
it . P
rent
ice
Hal
l,E
ngle
woo
d C
liffs
, NJ
1989
.
Pepp
er, A
llan
D.,
Man
agin
g th
e tr
aini
ng a
nd d
evel
opm
entf
unct
ion.
Ald
ersh
ot, H
ants
Eng
land
, Gow
er 1
984.
Rob
inso
n, D
ana,
Tra
inin
g Fo
r Im
pact
: H
ow to
link
trai
ning
tobu
sine
ss n
eeds
and
mea
sure
the
resu
lts. J
osey
-Bas
s Sa
n Fr
ncis
co
278
279
