Evaluation of the project P.A.T.H.S. based on students...
Transcript of Evaluation of the project P.A.T.H.S. based on students...
Title Evaluation of the project P.A.T.H.S. based on students' weeklydiaries: Findings from eight datasets
Author(s) Shek, DTL; Sun, RCF
Citation TheScientificWorldJournal, 2012, v. 2012, article no. 354254
Issued Date 2012
URL http://hdl.handle.net/10722/152905
Rights This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
The Scientific World JournalVolume 2012, Article ID 354254, 14 pagesdoi:10.1100/2012/354254
The cientificWorldJOURNAL
Research Article
Evaluation of the Project P.A.T.H.S. Based on Students’ WeeklyDiaries: Findings from Eight Datasets
Daniel T. L. Shek1, 2, 3, 4, 5 and Rachel C. F. Sun6
1 Department of Applied Social Sciences, The Hong Kong Polytechnic University Kowloon, Hong Kong2 Public Policy Research Institute, The Hong Kong Polytechnic University Kowloon, Hong Kong3 Department of Social Work, East China Normal University, Shanghai, China4 Kiang Wu Nursing College of Macau, Macau5 Division of Adolescent Medicine, Department of Pediatrics, Kentucky Children’s Hospital, University of Kentucky College of Medicine,Lexington, KY 40506-9983, USA
6 Faculty of Education, The University of Hong Kong Pokfulam, Hong Kong
Correspondence should be addressed to Daniel T. L. Shek, [email protected]
Received 5 November 2011; Accepted 25 December 2011
Academic Editor: Joav Merrick
Copyright © 2012 D. T. L. Shek and R. C. F. Sun. This is an open access article distributed under the Creative CommonsAttribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work isproperly cited.
This paper aims to investigate the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. (positive adolescent training throughholistic social programmes) based on eight datasets collected between 2005 and 2009. A total of 1,138 students who participatedin the program were randomly invited (from the whole grade or in some classes) to write a piece of journal in the form of aweekly diary in order to reveal their perceptions and feelings regarding the program and the perceived benefits of the program.Based on an integration of findings from different databases, results showed that the respondents generally (1) had positive viewson the program, (2) had positive views on the instructors, and (3) perceived that they had acquired competencies at the societal,school, familial, interpersonal, and personal levels after joining the program. Acknowledging the limitations of diaries, the presentqualitative findings provide support for the effectiveness of the Tier 1 Program of the Project P.A.T.H.S. in Hong Kong.
1. Introduction
The purpose of this paper is to report evaluation findingsof a positive youth development program in Hong Kong(Project P.A.T.H.S., i.e., positive adolescent training throughholistic social programmes) based on weekly diaries writtenby students participating in the program. The personal diaryis an important source of information to understand thesocial reality. For example, Anne Frank’s diary documentsthe thoughts, feelings, and experiences of a Jewish teenagegirl who had fled and hid in Amsterdam with her familywhen Nazi forces conquered The Netherlands. In view ofthe invaluable information gathered from people’s diaries,diaries for analyzing human behavior, discovering history,or evaluating a program represent an attractive researchapproach. In the context of social sciences research, diariesare commonly used in nonexperimental research to collectdata on human behavior. For example, Leigh [1] asked
participants to record their time, quantity, and type ofdrinking and sexual activity over 10 weeks in order toexamine the temporal relationships of alcohol consumptionto sexual behavior. Behavioral diaries have also been used inbehavioral modification programs [2]. There are studies tosuggest the strengths and weaknesses of using daily diariesas a research strategy [3, 4], and the debate has continued inthe literature [5, 6].
In the field of education, it is not uncommon to find thata teacher would keep reflective journals as a means to assesspersonal teaching performance, identify one’s pedagogicalstrengths and weaknesses, and evaluate the effectiveness ofthe changes implemented in teaching [7]. It is believed thata reflective journal “suggests question, identifies new areas toexplore, reveals meaningful absences, and uncovers recurringpatterns” ([8], page 16). Therefore, some studies utilizedteachers’ reflective diaries as a kind of data collection methodto investigate the essential components of effective teaching
2 The Scientific World Journal
practice or pedagogy, and to examine ways of contributing toprofessional development [9].
Similarly, in contrast to the traditional learning modelwhere students mainly take a passive role in receivingwhat their teachers’ transmit or reciting the informationfrom books, students are perceived as active learners whoconstruct their personal knowledge based on their ownexperiences in reflective learning models, where they areencouraged to be autonomous learners who reflect on theirlearning and to have more awareness on their learning pro-cesses, competencies, and limitations. Therefore, reflectiveobservation is highlighted to be an indispensable process intranslating concrete experiences to concepts and actions inthe experiential learning process [10], in which keeping adiary is considered as an opportunity for personal reflectionthat, in turn, enhances students’ autonomy and develops anownership of their learning process [11].
In order to guide students to become autonomouslearners, students studying in human services programs werecommonly asked to write diaries or reflective journals in dif-ferent ways. First, some courses simply aimed to encouragestudents to be critical and reflective learners. For example,in a module on management education, students were askedto write eight diaries with predesigned topics throughoutthe course [12]. Second, some courses were targeted towardsstrengthening the students’ ability to integrate their courselearning into actual field settings. For example, social workstudents were required to write reflective journals withpredesigned questions at several stages of their field work,on top of other written assignments [13]. Third, studentswere guided to be determinant masters of their future.For example, students in a health-discipline course wereasked to write a journal with no specific guidelines ontheir experiences of their practices in rural areas, so as toidentify the positive and negative aspects of rural practicesand develop their interest in working in the rural areas in thefuture [14]. Chaloner [15] affirmed that “using a personaljournal implies that the learner will construct a memory ofthe learning experience and how it can be applied to theworkplace by using a personal narrative. Through such anindividualized and personal approach, it is hoped that thelearner will be motivated enough to assume the autonomythat he/she is being offered” (page 22).
Besides understanding the subjective world views of theinformants, a survey of the literature in the field of evaluationshows that there is an increasing number of studies usingdiaries as a tool for evaluation. Wagner [16] used studentjournals to evaluate the effectiveness of a course. Follicket al. [17] evaluated a daily activity diary for chronic painpatients. Travers [18] examined the use of a reflective diarymethodology for exploring the lived experiences of stressand coping among 30 college students. Cohen et al. [19]examined the development and use of an online diary in theevaluation of a preventive care program that provided real-time communication between the evaluator and participants.Knowles and Tarrier [20] evaluated diary intervention onlevels of anxiety and depression in a group of intensive careunit survivors and demonstrated the therapeutic effects ofdiaries.
The main purpose of this paper is to report evaluationfindings of the Project P.A.T.H.S. using student diaries as anevaluation method. The project P.A.T.H.S. is a positive youthdevelopment program that attempts to promote holisticdevelopment among adolescents in Hong Kong, with aninitial earmarked grant of HK$400 million by The HongKong Jockey Club Charities Trust. The Project is divided intotwo tiers of programs. The Tier 1 Program is a universalcurriculum-based program for secondary 1 to 3 students inparticipating schools and the Tier 2 Program is a selectiveprogram for students with greater psychosocial needs. Incontrast to mainstream approaches that focus on tacklingyouth developmental problems, the field of positive youthdevelopment focuses on enabling the talents, strengths,interests, and future potentials in children and adolescents.Roth et al. [21] pointed out that positive youth developmentprograms are commonly regarded as “programs that provideopportunities and support to help youth gain the competen-cies and knowledge they need to meet the increasing chal-lenges they will face as they mature” (page 423). Althoughmany western programs have been developed to promotepositive development in children and adolescents, not all ofthem are successful in promoting adolescent development.For example, Catalano et al. [22] reviewed 77 programson positive youth development, and their review showedthat only 25 programs were successful. Obviously, evaluationis an important issue to be considered when assessing theeffectiveness of positive youth development programs.
To understand the program effects of the ProjectP.A.T.H.S., several complementary program evaluationstrategies, including both quantitative and qualitativemethodologies, have been adopted [23]. These includedobjective outcome evaluation, subjective outcome evalua-tion, process evaluation, interim evaluation, and qualitativeevaluation based on focus groups, in-depth interviews, andcase studies. Through the adoption of different evaluationstrategies, there is multiple evidence supporting the effec-tiveness of the Tier 1 Program of the Project P.A.T.H.S.based on different types of data collected from differentparticipants utilizing different methods. For example, bothquantitative findings [24, 25] and qualitative findings [26,27] based on different studies consistently suggest that theProject P.A.T.H.S. is helpful in promoting the psychosocialcompetencies of the program participants.
To further understand the perceived program effects ofthe Tier 1 Program, evaluation findings based on weeklydiaries written by students are reported in this paper.Actually, there are published papers on the evaluation of theProject P.A.T.H.S. based on weekly diaries [28, 29]. In thispaper, the findings based on weekly diaries in eight datasetswere aggregated in order to give a picture regarding theeffectiveness of the Project P.A.T.H.S. in Hong Kong. In short,the purpose of this paper is to give a descriptive picture onthe views and experiences of the program participants.
2. Methods
2.1. Participants and Procedures. From 2005 to 2009, a totalof 244 secondary schools participated in the P.A.T.H.S.
The Scientific World Journal 3
Table 1: Number of respondents writing weekly diaries.
S1 S2 S3
Study 12005/2006
EIP
Study 22006/2007
FIP
Study 32008/2009
FIP
Study 42006/2007
EIP
Study 52007/2008
FIP
Study 62008/2009
FIP
Study 72007/2008
EIP
Study 82008/2009
FIP
Total schools that joinedP.A.T.H.S.
52 207 197 49 196 198 48 167
(i) 10-hour program 23 95 104 27 113 110 29 104
(ii) 20-hour program 29 112 93 22 83 88 19 63
Total schools that joined thisstudy
4 6 2 4 6 2 4 6
(i) 10-hour program 1 0 0 0 0 0 0 0
(ii) 20-hour program 3 6 2 4 6 2 4 6
No. of diaries collected 95 216 58 260 164 62 132 151
Percentage of informants inthe selected schools
15% 19% 15% 36% 15% 16% 24% 16%
Note: S1: secondary 1 level; S2: secondary 2 level; S3: secondary 3 level; EIP: experimental implementation phase; FIP: full implementation phase.
Project. Among all the participating schools, 12 schoolsthat joined the secondary 1 level of the Project, 12 schoolsthat joined the secondary 2 level, and 10 schools thatjoined the secondary 3 level were randomly selected andinvited to take part in eight research studies. In each ofthe selected schools, the school contact person was askedto randomly invite some students (from the whole gradeor in some classes) to write a piece of journal in the formof a weekly diary in order to reveal their perceptions andfeelings after joining the Tier 1 Program of the project. Thestudents were informed of the purpose and confidentialityof this study, and their consent for participation was soughtprior to data collection. The title of the reflective journalwas “Participating in the Tier 1 Program of the ProjectP.A.T.H.S.: Experiences and Feelings.” The reflection wasexpected to be not less than 200 words in Chinese, and itshould be related to the students’ experiences, feelings, andcomments in connection with their participation in the Tier1 Program. The students could either complete it at homeor during class time. The total number of students’ diariesreceived was 1,138, comprising 369 written by secondary1 students, 486 written by secondary 2 students, and 283written by secondary 3 students (see Table 1). As the caseswere randomly selected and the response rate was 100%,the recruited sample could be regarded as respectable. Nosecondary 1 students were recruited in the 2007/2008 schoolyear because several cohorts of data had been collected.
While the basic research design of the studies generatingthe datasets was primarily qualitative in nature (i.e., writtencomments on the program, instructors, and benefits), itis possible to use quantitative methods (such as countingand frequency analyses) to create a descriptive profile ofthe findings [28]. In the present study, the secondarydata analysis approach was adopted by integrating andsynthesizing the data generated from the different databases.
2.2. Data Analyses. In each of the eight studies, the datawere analyzed using general qualitative analyses techniques
involving three steps [30]. First, relevant raw codes weredeveloped for words, phrases, and/or sentences that formedmeaningful units at the raw responses level. Second,the codes were further combined to reflect higher-orderattributes at the category of codes level (i.e., themes andsubthemes). Third, the categories of codes were furtheranalyzed to reveal the broader themes at the thematiclevel. Three main themes were identified from the weeklydiaries: views on the program, views on the instructors,and perceived benefits of the program. Both intra- andinterrater reliabilities on the coding regarding these viewswere calculated, and the results were satisfactory (85–100%).In the present analyses, because the researchers designedthe program of the Project P.A.T.H.S., they were consciousof their own biases and expectations on the program tobe effective. This paper reports the analyses based onthe aggregated findings of these eight studies. The majorcategories and subcategories of responses are presented in thetables, and narrative illustrating the themes can be seen in theSection 3.
3. Results
Responses regarding the students’ perception of the programand the instructor, as well as the perceived benefits of theprogram, are presented in this paper. The integrated findingsare presented in Tables 2–4. As shown in Table 2, there wasa total of 2,290 meaningful units regarding students’ viewson the Tier 1 Program implemented from 2005 to 2009. Theresponses were categorized into “overall impression,” “pro-gram content,” “learning process,” “program arrangement,”and “other comments.” About 81% of the total responses wascoded as positive. Most of the respondents had positive viewson the “overall impression,” followed by “learning process”and “program content”. There were 85.4, 77.2, and 83.1% ofsecondary 1, 2, and 3 students who reported positively onthe program, respectively, although some negative commentswere recorded. Three cases illustrating the respondents’
4 The Scientific World Journal
Ta
ble
2:V
iew
son
the
prog
ram
acro
ssst
udy
1to
stu
dy8.
Vie
ws
onth
epr
ogra
mR
espo
nse
sS1
S2S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
P
Ove
rall
impr
essi
on
Posi
tive
:(e
.g.,
mea
nin
gfu
l,ex
celle
nt,
ben
efici
al,a
nd
liked
the
prog
ram
)
2611
643
274
3750
5910
671
1
Neg
ativ
e:(e
.g.,
mea
nin
gles
s,ba
d,di
sapp
oin
ted,
and
not
inte
rest
ing)
01
518
97
28
50
Neu
tral
:(e
.g.,
mix
edbl
essi
ngs
,fai
r,n
otdi
ffer
ent
from
oth
erpr
ogra
ms,
and
sim
ilar
toth
esu
bjec
tsin
prim
ary
sch
ools
)
30
03
01
03
10
Subt
otal
2911
748
295
4658
6111
777
1
Pro
gram
con
ten
t
Posi
tive
:(e
.g.,
prac
tica
l,cr
eati
ve,r
ich
con
ten
t,an
dea
sily
un
ders
tood
)27
2535
6831
4378
6637
3
Neg
ativ
e:(e
.g.,
disl
iked
grow
thpu
zzle
s,bo
rin
gco
nte
nt,
too
man
yw
orks
hee
ts,a
nd
diss
atis
fied
wit
hth
eac
tivi
ties
)
630
108
2649
2421
174
Neu
tral
:(e
.g.,
com
men
tson
teac
hin
gm
ater
ials
)0
00
00
00
22
Subt
otal
3355
4576
5792
102
8954
9
Lear
nin
gpr
oces
s
Posi
tive
:(e
.g.,
fun
ny,i
nte
rest
ing,
hap
py,
and
acti
vepa
rtic
ipat
ion
)34
7025
107
6226
4810
247
4
Neg
ativ
e:(e
.g.,
bori
ng,
un
invo
lved
,poo
rle
arn
ing
atm
osph
ere,
and
non
sen
se)
518
296
03
333
160
Subt
otal
3988
2720
362
2981
105
634
The Scientific World Journal 5
Ta
ble
2:C
onti
nu
ed.
Vie
ws
onth
epr
ogra
mR
espo
nse
sS1
S2S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
P
Pro
gram
arra
nge
men
t
Posi
tive
:(e
.g.,
good
deliv
ery
mod
e,go
odar
ran
gem
ent,
and
rew
ard
sch
eme)
00
50
110
026
42
Neg
ativ
e:(e
.g.,
inad
equ
ate
reso
urc
es,p
oor
arra
nge
men
t,in
suffi
cien
tti
me,
and
bad
clas
sroo
mm
anag
emen
t)
00
00
54
67
22
Subt
otal
00
50
614
633
64
Oth
erco
mm
ents
Posi
tive
2355
797
95
4117
254
Neg
ativ
e0
40
60
00
111
Neu
tral
00
07
00
00
7Su
btot
al23
597
110
95
4118
272
Gra
nd
tota
lPo
siti
ve(%
)11
0(8
8.7)
266
(83.
4)11
5(8
7.1)
546
(79.
8)14
0(7
7.8)
134
(67.
7)22
6(7
7.7)
317
(87.
6)18
54N
egat
ive
(%)
11(8
.9)
53(1
6.6)
17(1
2.9)
128
(18.
7)40
(22.
2)63
(31.
8)65
(22.
3)40
(11)
417
Neu
tral
(%)
3(2
.4)
00
10(1
.5)
01
(0.5
)0
5(1
.4)
19To
tal(
%)
124
(100
)31
9(1
00)
132
(100
)68
4(1
00)
180
(100
)19
8(1
00)
291
(100
)36
2(1
00)
2290
Tota
lin
each
grad
ePo
siti
ve(%
)49
1(8
5.4)
820
(77.
2)54
3(8
3.1)
1854
Neg
ativ
e(%
)81
(14.
1)23
1(2
1.8)
105
(16.
1)41
7N
eutr
al(%
)3
(0.5
)11
(1)
5(0
.8)
19To
tal(
%)
575
(100
)10
62(1
00)
653
(100
)22
90
6 The Scientific World Journal
Ta
ble
3:V
iew
son
the
inst
ruct
orac
ross
stu
dy1
tost
udy
8.
Vie
ws
onth
ein
stru
ctor
Res
pon
ses
S1S2
S3
Tota
lSt
udy
120
05/2
006
EIP
Stu
dy2
2006
/200
7FI
P
Stu
dy3
2008
/200
9FI
P
Stu
dy4
2006
/200
7E
IP
Stu
dy5
2007
/200
8FI
P
Stu
dy6
2008
/200
9FI
P
Stu
dy7
2007
/200
8E
IP
Stu
dy8
2008
/200
9FI
P
Ove
rall
impr
essi
on
Posi
tive
:(e
.g.,
nic
e,lik
edin
stru
ctor
s,co
uld
mot
ivat
est
ude
nts
,an
dth
ankf
ult
oin
stru
ctor
s)
28
041
26
2216
97
Neg
ativ
e:(e
.g.,
disg
ust
ing,
stri
ct,
alw
ays
blam
ing,
and
inco
nsi
sten
tfa
cilit
atio
nsk
ills
betw
een
inst
ruct
ors)
31
03
00
00
7
Subt
otal
59
044
26
2216
104
Teac
hin
gpe
rfor
man
ce
Posi
tive
:(e
.g.,
exce
llen
t,w
ell
prep
ared
,pro
fess
ion
al,a
nd
shar
edow
nex
peri
ence
s)
928
1751
2817
2745
222
Neg
ativ
e:(e
.g.,
bori
ng
teac
hin
g,po
orcl
assr
oom
man
agem
ent,
poor
teac
hin
gsk
ills,
and
tau
ght
ina
hurr
ym
ann
er)
03
04
30
231
34
Subt
otal
931
1755
3117
5046
256
Teac
hin
gat
titu
de
Posi
tive
:(e
.g.,
enth
usi
asti
c,de
vote
d,w
illin
g/ab
leto
hel
pst
ude
nts
,an
dpa
tien
t)
57
440
1012
2757
162
Subt
otal
57
440
1012
2757
162
Oth
ers
Posi
tive
33
015
00
40
25N
egat
ive
00
03
00
20
5N
eutr
al0
02
20
20
06
Subt
otal
33
220
02
60
36G
ran
dto
tal
Posi
tive
(%)
19(8
6)46
(92)
21(9
1.3)
147
(92.
4)40
(93)
35(9
4.6)
80(7
6.2)
118
(99.
2)50
6N
egat
ive
(%)
3(1
4)4
(8)
010
(6.3
)3
(7)
025
(23.
8)1
(0.8
)46
Neu
tral
(%)
00
2(8
.7)
2(1
.3)
02
(5.4
)0
06
Tota
l(%
)22
(100
)50
(100
)23
(100
)15
9(1
00)
43(1
00)
37(1
00)
105
(100
)11
9(1
00)
558
Tota
lin
each
grad
ePo
siti
ve(%
)86
(90.
5)22
2(9
2.9)
198
(88.
4)50
6
Neg
ativ
e(%
)7
(7.4
)13
(5.4
)26
(11.
6)46
Neu
tral
(%)
2(2
.1)
4(1
.7)
06
Tota
l(%
)95
(100
)23
9(1
00)
224
(100
)55
8
The Scientific World Journal 7
Ta
ble
4:Pe
rcei
ved
prog
ram
effec
tive
nes
sac
ross
stu
dy1
tost
udy
8.
Eco
logi
call
evel
Subd
omai
ns
Des
crip
tion
s
S1S2
S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
P
Soci
etal
Soci
aln
orm
sFo
ster
edpr
osoc
ialn
orm
sor
invo
lvem
ent
326
27
98
07
62
Soci
alre
spon
sibi
lity
and
affai
rs
En
han
ced
know
ledg
eab
out
Ch
ina
and
Hon
gK
ong
69
01
00
00
16
En
han
ced
know
ledg
eab
out
publ
icaff
airs
10
00
00
00
1
Rec
ogn
ized
citi
zen’
sre
spon
sibi
litie
s4
100
30
00
017
Subt
otal
1445
211
98
07
96
Sch
ool
Sch
ool
belo
ngi
ngn
ess
Incr
ease
dbe
lon
gin
gnes
sto
sch
ool
00
00
00
10
1
Subt
otal
00
00
00
10
1
Fam
ilial
Fam
ilyre
lati
onsh
ips
En
han
ced
rela
tion
ship
sw
ith
pare
nts
and
oth
erfa
mily
mem
bers
522
1912
104
111
84
Subt
otal
522
1912
104
111
84
Inte
rper
son
alG
ener
alin
terp
erso
nal
com
pet
ence
En
han
ced
team
spir
it9
150
180
00
042
Mad
efr
ien
ds7
150
60
00
028
Got
alon
gw
ith
oth
ers
1965
041
00
00
125
En
han
ced
peer
rela
tion
ship
028
110
2312
1217
103
En
han
ced
rela
tion
ship
wit
hin
stru
ctor
s/so
cial
wor
kers
02
00
45
48
23
Abl
eto
diff
eren
tiat
ebe
twee
nh
ealt
hyan
du
nh
ealt
hyfr
ien
ds7
220
24
00
035
Abl
eto
han
dle
love
affai
rs0
00
40
00
04
Stre
ngt
hen
edso
cial
bon
din
g0
00
220
08
030
En
han
ced
soci
alco
mp
eten
ce0
170
210
00
038
Tota
lin
subc
ateg
ory
4216
411
114
3117
2425
428
Spec
ific
inte
rper
son
alco
mp
eten
ceIm
prov
edin
terp
erso
nal
skill
s0
014
08
70
1948
Impr
oved
com
mu
nic
atio
nsk
ills
330
821
1615
98
110
En
han
ced
pres
enta
tion
abili
ty0
00
150
00
015
Lear
ned
toac
cept
,car
e,tr
ust
,re
spec
t,an
du
nde
rsta
nd
oth
ers
338
1519
117
1318
124
8 The Scientific World Journal
Ta
ble
4:C
onti
nu
ed.
Eco
logi
call
evel
Subd
omai
ns
Des
crip
tion
s
S1S2
S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
PP
rom
oted
coop
erat
ion
wit
hot
her
s0
44
019
110
442
Incr
ease
dw
illin
gnes
sto
expr
ess
00
00
30
30
6To
tali
nsu
bcat
egor
y6
7241
5557
3035
4934
5Su
btot
al48
236
5216
988
4759
7477
3
Pers
onal
Gen
eral
gain
sP
rom
oted
stu
den
ts’
deve
lopm
ent
00
20
324
3510
83
Acq
uir
edsk
ills/
know
ledg
e14
266
3128
90
2914
3Le
arn
edso
met
hin
gu
sefu
lan
dap
plic
able
toda
ilylif
e13
180
00
00
435
Hel
pfu
lto
stu
den
ts0
013
04
80
1742
Posi
tive
chan
ges
onst
ude
nts
00
00
00
09
9To
tali
nsu
bcat
egor
y27
4421
3164
2135
6931
2B
ehav
iora
lco
mp
eten
ceIm
prov
edre
fusa
lski
lls0
110
00
00
011
Impr
oved
deci
sion
-mak
ing
skill
s0
00
00
70
815
Lear
ned
fin
anci
alm
anag
emen
tsk
ills
00
00
00
53
8
En
han
ced
tim
em
anag
emen
tsk
ills
00
00
10
20
3
En
han
ced
lead
ersh
ipsk
ills
00
00
00
10
1G
ener
albe
hav
iora
lcom
pet
ence
00
20
153
84
32To
tali
nsu
bcat
egor
y0
112
016
1016
1570
Pers
onal
Cog
nit
ive
com
pet
ence
Pro
mot
edra
tion
alan
dan
alyt
ical
thin
kin
g5
180
260
00
049
En
han
ced
crea
tivi
ty0
20
02
00
04
Impr
oved
prob
lem
-sol
vin
gsk
ills
726
1322
147
915
113
Gen
eral
cogn
itiv
eco
mp
eten
ce0
04
622
513
2070
Tota
lin
subc
ateg
ory
1246
1754
3812
2235
236
The Scientific World Journal 9
Ta
ble
4:C
onti
nu
ed.
Eco
logi
call
evel
Subd
omai
ns
Des
crip
tion
s
S1S2
S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
PM
oral
com
pet
ence
Cu
ltiv
ated
per
son
alet
hic
s/vi
rtu
es2
66
250
30
1860
En
han
ced
resp
onsi
bilit
y0
00
70
00
07
En
han
ced
empa
thy
03
06
00
00
9P
rom
oted
the
idea
ofeq
ual
ity
and
fair
nes
s3
00
00
00
03
Abl
eto
dist
ingu
ish
betw
een
righ
tan
dw
ron
g5
543
190
30
791
Abl
eto
corr
ect
bad
hab
its/
defe
cts
414
03
00
00
21
Cu
ltiv
ated
prop
erat
titu
des
tow
ards
sex
and
love
rela
tion
ship
00
00
40
410
18
Gen
eral
mor
alco
mp
eten
ce0
00
04
015
625
Tota
lin
subc
ateg
ory
1477
960
86
1941
234
Em
otio
nal
com
pet
ence
En
han
ced
emot
ion
alm
anag
emen
t/ex
pres
sion
1157
2728
99
1017
168
Tota
lin
subc
ateg
ory
1157
2728
99
1017
168
Bel
iefs
inth
efu
ture
Pro
mot
edbe
liefs
inth
efu
ture
00
00
01
512
18
Ass
ista
nce
inca
reer
plan
nin
g0
00
00
00
44
Faci
litat
edgo
alse
ttin
g6
234
295
49
989
Tota
lin
subc
ateg
ory
623
429
55
1425
111
Res
ilien
ceC
ult
ivat
edre
silie
nce
00
424
1315
1020
86O
ptim
ism
15
09
04
00
19W
ays
tofa
cead
vers
ity
012
00
00
00
12To
tali
nsu
bcat
egor
y1
174
3313
1910
2011
7
10 The Scientific World Journal
Ta
ble
4:C
onti
nu
ed.
Eco
logi
call
evel
Subd
omai
ns
Des
crip
tion
s
S1S2
S3
Tota
lSt
udy
120
05/0
6E
IPSt
udy
220
06/0
7FI
PSt
udy
320
08/0
9FI
PSt
udy
420
06/0
7E
IPSt
udy
520
07/0
8FI
PSt
udy
620
08/0
9FI
PSt
udy
720
07/0
8E
IPSt
udy
820
08/0
9FI
PPo
siti
vese
lfFa
cilit
ated
per
son
algr
owth
1142
034
00
00
87D
evel
oped
clea
ran
dpo
siti
veid
enti
ty0
00
00
10
12
En
han
ced
self
-effi
cacy
00
42
33
05
17E
nh
ance
dse
lf-c
onfi
den
ce3
270
92
314
967
En
han
ced
self
-det
erm
inat
ion
014
015
21
42
38Fa
cilit
ated
self
-un
ders
tan
din
g18
417
2818
530
1316
0D
evel
oped
pote
nti
als/
wid
ened
hor
izon
00
00
00
07
7
Tota
lin
subc
ateg
ory
3212
411
8825
1348
3737
8C
her
ish
ing
life
Lear
ned
toch
eris
hlif
e1
30
419
82
1377
Cu
ltiv
ated
posi
tive
life
view
s/le
arn
edlif
epr
inci
ples
00
30
014
039
56
Lear
ned
totr
easu
rep
eopl
ean
dth
ings
121
012
00
90
43
Self
-refl
ecti
on3
97
93
311
1863
Pro
mot
edre
flec
tion
oflif
e5
50
2812
130
871
Tota
lin
subc
ateg
ory
1038
1090
2438
2278
310
Subt
otal
113
437
105
413
202
133
196
337
1936
Oth
ers
Posi
tive
com
men
tsIn
tere
stin
lear
nin
g1
40
102
00
017
Stim
ula
tin
g2
00
00
00
02
Fru
itfu
l10
00
00
00
010
Aca
dem
icim
prov
emen
ts0
00
00
00
44
Hea
lthy
livin
gst
yle
00
012
20
00
14O
ther
s10
84
80
60
440
Tota
lin
subc
ateg
ory
2312
430
46
08
87N
egat
ive
com
men
tsP
rogr
amw
asn
oteff
ecti
ve0
01
00
20
03
Tota
lin
subc
ateg
ory
00
10
02
00
3
Oth
ers
Neu
tral
com
men
tsC
ould
not
hel
pm
uch
onst
ude
nts
’dev
elop
men
t0
00
00
03
03
Oth
ers
00
00
00
03
3To
tali
nsu
bcat
egor
y0
00
00
03
36
Subt
otal
2312
530
48
311
96G
ran
dto
tal
203
752
183
635
313
200
260
440
2986
The Scientific World Journal 11
perceptions of the program at each grade are presentedbelow.
Student A (Secondary 1, 2006/2007). “This is my first timejoining the P.A.T.H.S. Project. I learned about emotion,choice and other knowledge about development from theprogram . . . I personally think that this program has greatimpact on me and my classmates. We could learn via theworksheet activities as well as from the advice given by theclassmates. What we learned are closely related to realitywhich could be applicable to daily life. Therefore, I think theprogram is very meaningful.”
Student B (Secondary 2, 2007/2008). “After completion ofthe Tier 1 Program in the first semester, I felt I becamemore mature in interpersonal communication and personalgrowth. We studied various problems such as teenagers’learning attitudes and hedonism during class. Throughgames or group discussion, we learned how to analyze andacquire correct information, and how to face difficultiesand remain positive under adverse circumstances. I like thisprogram very much because all the program content isclosely related to common teenage problems, such as bodyshaping and substance abuse, which is closely related to ourdaily life. If we do not have good virtues and healthy mind,we will follow the wrong track easily. Therefore, the programserves as not only a light in the dark but also a teacher whoteaches and guides us to the right way.”
Student C (Secondary 3, 2008/2009). “Fortunately, I joinedthe Project P.A.T.H.S. from Secondary 1 to Secondary 3.This is one of my favorite subjects. The atmosphere duringthe class was relaxing that made the subject become moreacceptable. Moreover, I really liked the design of the program.Teachers and social workers used multiple resources forteaching in order to guide the students to think fromdifferent angles and make judgment. Problems such as schoolorientation, conflicts with parents and peer relationship thattroubled me a lot were discussed in the class. I could solvethese problems by applying the skills I learned from the class.Obviously the project works.”
There were 558 responses regarding “views on instruc-tors,” which were categorized into “overall impression,”“teaching performance,” “teaching attitude,” and “others”(see Table 3). About 90% was positive responses. Most ofthe respondents had positive views on instructors’ “teachingperformance.” Several cases illustrating the respondents’views on the instructors are shown below.
Student D (Secondary 1, 2006/2007). “I remembered whenI first joined the Project P.A.T.H.S., I felt very excited . . .Miss Wong was our instructor. Although the topics of eachP.A.T.H.S. session were different, the goal was the same—to help students to develop good interpersonal relationship.The instructor would encourage the students to share theiropinions on a topic and then asked other students to discussor analyze each student’s viewpoints. The instructor wouldalso arrange some activities such as group discussion and
role-play. These activities promoted the relationship amongclassmates in only one year’s time.”
Student E (Secondary 3, 2008/2009). “There was a period oftime when I lost self-confidence due to a failure. I kept myselfin the corner of my world and could not find a way out untilI heard some sharing from the instructor. She said, “Life islike an obstacle race. No matter what obstacles you encounterduring the race, once you pass the finishing line, you willhave victory.” I suddenly realized that self-pitying did nothelp anything. Rather, I should regain my strength to facenew challenges. The words from the instructor become mymotto, which drives me to keep moving forward.”
The perceived benefits of the program to the students areshown in Table 4. The 2,986 meaningful units were catego-rized into six main categories (i.e., societal, school, familial,interpersonal, personal levels of competence, and others).Most of the responses were related to the effectiveness of theprogram on a personal level (n = 1,936), particularly in thesubcategory of “positive self” (n = 378, e.g., “enhanced self-understanding”), followed by “general gains” (n = 312, e.g.,“acquired skills/knowledge”) and “cherishing life” (n = 310,e.g., “promoted reflection of life”). In addition, many respon-dents reported that they acquired interpersonal competence(n = 773), which could be categorized into “general inter-personal competence” (e.g., “got along with others”) and“specific interpersonal competence” (e.g., “learned to accept,care, trust, respect, and understand others”). Some narrativesbased on the respondents revealed that the program wasperceived positively by the program participants.
Student F (Secondary 1, 2006/2007). “I learned much usefulknowledge in this program. Firstly, I learned many interper-sonal skills such as how to get along with classmates, teachers,friends, senior people and family members. I also learnedto appreciate others and myself. I knew myself much morethan before. By realizing my strengths and weaknesses, I wasable to improve myself. In addition, I learned to distinguishbetween what I should do and what I should not. Mostimportant of all, the program enabled me to meet newfriends and improve my communication skills. All these skillswill be very useful for my future development.”
Student G (Secondary 2, 2007/2008). “When I looked back,I really learned a lot of things. I felt I truly grew up. Infamilial aspect, I understood that family members shouldmutually help each other and care for one another. I need tohelp doing the family chores when I have time. As parentshave been taking care of us whole-heartedly, we have torepay and be thankful to them. I learned the importance ofcommunication. Now I would spend more time to chat withmy parents and share my feelings with them. In doing so,my parents were reassured and became less worried aboutme . . . In social aspect, I knew that we should follow socialdisciplines such as keeping the voice down in public area toavoid interrupting other people . . . In global aspect, I learnedthat there were many people in the world living in poverty.We should do whatever we could do to help these people.”
12 The Scientific World Journal
Student H (Secondary 3, 2008-2009). “I could still rememberwhen I first participated in the Project P.A.T.H.S., I didnot understand why we should sacrifice two sessions oflesson for studying this program. However, I was graduallyinvolved in it. This program provided me an opportunity toknow more about my teachers and classmates . . . Althoughthe three-year program is going to end, we still have alot to learn throughout our lives. Among the units I havelearned, the most impressive topic is “Life and Death”. “Lifeand Death” is actually not far away from us. Rememberthe Wenchuan earthquake last year? Life of the students inSichuan was changed in only one second. As Hong Kongstudents, we should make deepest reflection—bliss is nottaken for granted!”
4. Discussion
The purpose of this study was to evaluate the Tier 1 Programof the Project P.A.T.H.S. using findings based on weeklydiaries collected in the experimental and full implementationphases (2005–2009) of the project. There are several char-acteristics of this study. First, a large sample of participants(n = 1,138 students) was involved. Second, eight datasetscollected across three grades at different points in time wereanalyzed. Third, this is the first known scientific evaluationstudy of a positive youth development program utilizingstudent diaries based on different cohorts in China. Fourth,this is also the first evaluation study utilizing diaries basedon such a large sample of participants in the global context.Finally, the data were analyzed in both quantitative andqualitative ways. As far as the design and data collection ofthe studies are concerned, the study has the characteristics ofnaturalistic inquiry, inductive analysis, holistic perspective,qualitative data, unique case orientation, and contextualsensitivity. Regarding data analyses, both qualitative (e.g.,coding and categorizing) and quantitative (mainly counting)analyses were used.
Based on the weekly diaries written by the students, thestudies reported in this paper showed that the participantsgenerally had positive perceptions of the Tier 1 Programof the Project P.A.T.H.S. and the instructors. The findingspresented in Tables 2 and 3 suggest that the participants hadpositive impressions of the program, and they appreciatedthe program content and learning process involved. Inaddition, Table 4 shows that the participants overwhelminglyregarded the program as beneficial to the cognitive, emo-tional, social, and productive quality of life of the programparticipants. For example, a sizable number of responseswere related to positive self-image after joining the program(Table 4). In addition, many participants remarked that theirinterpersonal competencies were enhanced after joining theprogram. In short, the findings in Table 4 basically suggestthat the program is beneficial to the development of theprogram participants.
The above findings are generally consistent with previousresearch findings in the experimental and full implemen-tation phases, and they basically concur with both thequantitative and qualitative findings based on objective
outcome, subjective outcome, qualitative, and process eval-uation findings (e.g., [24–27, 31]). In short, the presentfindings provide additional evidence for supporting theeffectiveness of the Tier 1 Program of the Project P.A.T.H.S.in Hong Kong. From the perspective of triangulation, theexisting evaluation findings suggest that the effectiveness ofthe Tier 1 Program of the Project P.A.T.H.S. is supportedby evaluation data collected from different sources and bydifferent strategies.
The present findings demonstrate the utility of usingstudent diaries to evaluate positive youth developmentprograms. By asking the program participants to reflecton their experiences, subjective perceptions and perceivedbenefits of the program can be properly understood in aless mechanical and artificial manner. Such an approachcan help to capture the dynamic nature of the perceivedqualities and effectiveness of the program. In addition to itsholistic emphasis and ease of data collection, Maxwell [32]commented that “qualitative research is a rigorous meansof investigating causality” (page 3). Slayton and Llosa [33]also remarked that “qualitative methods can confirm thatit is actually the program that is responsible for the effect”(page 2544). Obviously, the use of reflective journals enjoysthe strengths of qualitative studies in assessing the changesin quality of life among the participants after joining theprogram.
Shek et al. [34] proposed that several principles shouldbe maintained in order to enhance the quality of qualitativeresearch. As a qualitative study, the following attributes areintrinsic to this study. First, a general qualitative orientationwas adopted. Second, recruitment processes for the partic-ipants and justifications for the number of participants aredescribed. Third, details of the data collection are given.Fourth, issues of biases and ideological preoccupation areaddressed. Fifth, inter- and intrarater reliabilities informa-tion is presented. Finally, the categorized data are kept in asystematic filing system in order to ensure that the findingsare auditable.
Nevertheless, several limitations of the study should benoted. First, for students who are sensitive about the issueof “invasion of privacy” [12], the use of a weekly diary maybe regarded as an obtrusive evaluation device. Second, asthe diary method is written in nature, students who arenot good at using words to express their experiences maywrite very little. This problem may be particularly acute inschools that admit students with poor academic performanceor literacy. This is also a problem in the Chinese culturewhere Chinese people may lack the language to describe theirfeelings. Third, as there was only one occasion on whichthe participants expressed their views in a one-way manner,it was not possible to have dialogues with the informantsto further understand some of their views. In particular,theoretical sampling approaches could be attempted infuture studies.
Fourth, although the present findings are interpretedin terms of the positive program effects and experiencesof the program participants, it should be noted thatthere are several alternative explanations. These alternativeexplanations should be considered because high proportions
The Scientific World Journal 13
of responses were positive across the eight studies. Thefirst alternative explanation is that the students were afraidof being punished by the program implementers if theydid not respond in a favorable manner. However, thispossibility can be partially dismissed because the teachersdid not mark the weekly diaries and the identities of thestudents were basically anonymous. The second alternativeexplanation is that the students consciously acted in a “nice”manner to help the workers to illustrate a positive programeffect. However, this alternative explanation can be partiallydismissed because negative comments were recorded andstudents are normally encouraged to express their viewshonestly. The third alternative explanation is that the highproportion of positive responses observed, were in fact,biased responses. However, this alternative explanation canalso be dismissed because the schools and students wererandomly selected. The fourth alternative explanation is thatbecause there is no control group, natural developmentalmaturation may account for the positive evaluation findings.That is, through maturation, the participants may see thingsin a more favorable manner. Furthermore, threats to internalvalidity in the study should be realized.
Finally, although the 11 principles proposed by Sheket al. [34] were upheld in this study, peer and memberchecking (principle 8) was not carried out because of timeand manpower constraints. Despite these limitations, thisstudy provides pioneering evaluation findings across samplesand time that support the positive nature of the ProjectP.A.T.H.S. and its effectiveness in promoting holistic youthdevelopment among Chinese adolescents in Hong Kong.
Acknowledgments
The preparation for this paper and the Project P.A.T.H.S.were financially supported by The Hong Kong Jockey ClubCharities Trust. The authorship is equally shared between thefirst and second authors.
References
[1] B. C. Leigh, “Alcohol consumption and sexual activity asreported with a diary technique,” Journal of Abnormal Psychol-ogy, vol. 102, no. 3, pp. 490–493, 1993.
[2] L. Peterson, G. Tremblay, B. Ewigman, and C. Popkey, “Theparental daily diary: a sensitive measure of the process ofchange in a child maltreatment prevention program,” BehaviorModification, vol. 26, no. 5, pp. 627–647, 2002.
[3] B. C. Leigh, “Using daily reports to measure drinking anddrinking patterns,” Journal of Substance Abuse, vol. 12, no. 1-2,pp. 51–65, 2000.
[4] D. M. Morrison, B. C. Leigh, and M. R. Gillmore, “Daily datacollection: a comparison of three methods,” Journal of SexResearch, vol. 36, no. 1, pp. 76–81, 1999.
[5] M. R. Gillmore, J. Gaylord, J. Hartway et al., “Daily datacollection of sexual and other health-related behaviors,”Journal of Sex Research, vol. 38, no. 1, pp. 35–42, 2001.
[6] A. A. Stone, R. C. Kessler, and J. A. Haythornthwaite,“Measuring daily events and experiences: decisions for theresearcher,” Journal of Personality, vol. 59, no. 3, pp. 575–607,1991.
[7] D. Gorman, “Self-tuning teachers: using reflective journals inwriting classes,” Journal of Adolescent and Adult Literacy, vol.41, no. 6, pp. 434–442, 1998.
[8] D. Hobson, “Beginning with the self: using autobiographyand journal writing in research,” in Teacher Doing Research:Practical Possibilities, G. Burnaford, J. Fischer, and D. Hobson,Eds., pp. 1–17, Erlbaum, Mahwah, NJ, USA, 1996.
[9] H. Bartholomew, J. Osborne, and M. Ratcliffe, “Teachingstudents “ideas-about-science”: five dimensions of effectivepractice,” Science Education, vol. 88, no. 5, pp. 655–682, 2004.
[10] D. A. Kolb, Experiential Learning, Prentice Hall, EnglewoodCliffs, NY, USA, 1984.
[11] E. L. Deci and R. Flaste, Why We Do What We Do: Understand-ing Self-Motivation, Penguins Books, New York, NY, USA,1996.
[12] A. Rothwell and S. Ghelipter, “The developing manager:reflective learning in undergraduate management education,”Reflective Practice, vol. 4, no. 2, pp. 241–254, 2003.
[13] N. Ames and S. A. Diepstra, “Using intergenerational oralhistory service-learning projects to teach human behaviorconcepts: a qualitative analysis,” Educational Gerontology, vol.32, no. 9, pp. 721–735, 2006.
[14] L. McAllister, E. McEwen, V. Williams, and N. Frost, “Ruralattachments for students in the health professions: are theyworthwhile?” The Australian Journal of Rural Health, vol. 6, no.4, pp. 194–201, 1998.
[15] B. Chaloner, “An evaluative study on building the bridgebetween the training room and the workplace,” EducationalResearch and Evaluation, vol. 19, no. 1, pp. 21–37, 2006.
[16] Z. M. Wagner, “Using student journals for course evaluation,”Assessment and Evaluation in Higher Education, vol. 24, no. 3,pp. 261–272, 1999.
[17] M. J. Follick, D. K. Ahern, and N. Laser-Wolston, “Evaluationof a daily activity diary for chronic pain patients,” Pain, vol.19, no. 4, pp. 373–382, 1984.
[18] C. Travers, “Unveiling a reflective diary methodology forexploring the lived experiences of stress and coping,” Journalof Vocational Behavior, vol. 79, no. 1, pp. 204–216, 2010.
[19] D. J. Cohen, L. C. Leviton, N. Isaacson, A. F. Tallia, and B.F. Crabtree, “Online diaries for qualitative evaluation: gainingreal-time insights,” American Journal of Evaluation, vol. 27, pp.163–184, 2006.
[20] R. E. Knowles and N. Tarrier, “Evaluation of the effectof prospective patient diaries on emotional well-being inintensive care unit survivors: a randomized controlled trial,”Critical Care Medicine, vol. 37, no. 1, pp. 184–191, 2009.
[21] J. Roth, J. Brooks-Gunn, L. Murray, and W. Foster, “Promotinghealthy adolescents: synthesis of youth development programevaluations,” Journal of Research on Adolescence, vol. 8, no. 4,pp. 423–459, 1998.
[22] R. F. Catalano, M. L. Berglund, J. A. M. Ryan, H. S. Lonczak,and J. D. Hawkins, “Positive youth development in the UnitedStates: research findings on evaluations of positive youthdevelopment programs,” Annals of the American Academy ofPolitical and Social Science, vol. 591, pp. 98–124, 2004.
[23] D. T. L. Shek and R. C. F. Sun, “Effectiveness of the tier 1program of project P.A.T.H.S.: findings based on three yearsof program implementation,” TheScientificWorldJournal, vol.10, pp. 1509–1519, 2010.
[24] D. T. L. Shek and C. M. S. Ma, “Impact of the ProjectP.A.T.H.S. in the junior secondary school years: individualgrowth curve analyses,” TheScientificWorldJournal, vol. 11, pp.253–266, 2011.
14 The Scientific World Journal
[25] D. T. L. Shek and L. Yu, “Prevention of adolescent problembehavior: longitudinal impact of the Project P.A.T.H.S. inHong Kong,” TheScientificWorldJournal, vol. 11, pp. 546–567,2011.
[26] D. T. L. Shek and C. Chan, “Qualitative evaluation of theproject P.A.T.H.S. based on junior secondary school studentsin Hong Kong,” International Journal of Adolescent Medicineand Health, vol. 22, no. 4, pp. 511–525, 2010.
[27] D. T. L. Shek and R. C. F. Sun, “Subjective outcome evaluationbased on secondary data analyses: the Project P.A.T.H.S. inHong Kong,” TheScientificWorldJournal, vol. 10, pp. 224–237,2010.
[28] D. T. L. Shek, “Using students’ weekly diaries to evaluatepositive youth development programs: are findings based onmultiple studies consistent?” Social Indicators Research, vol. 95,no. 3, pp. 475–487, 2010.
[29] D. T. L. Shek and R. C. F. Sun, “Using student weekly diaryto evaluate positive youth development programs: the caseof Project P.A.T.H.S. in Hong Kong,” International Journal ofAdolescent Medicine and Health, vol. 1, no. 2, pp. 93–102, 2008.
[30] M. B. Miles and A. M. Huberman, Qualitative Data Analysis,Sage, Thousand Oaks, Calif, USA, 1994.
[31] D. T. L. Shek, R. C. F. Sun, and C. S. M. Ng, “Processevaluation of the implementation of the Secondary 3 Programof Project P.A.T.H.S. in the Experimental ImplementationPhase,” International Public Health Journal, vol. 1, no. 3, pp.311–324, 2009.
[32] J. A. Maxwell, “Causal explanation, qualitative research, andscientific inquiry in Journal item: education,” EducationResearch, vol. 33, pp. 3–11, 2004.
[33] J. Slayton and L. Llosa, “The use of qualitative methods inlarge-scale evaluation: improving the quality of the evaluationand the meaningfulness of the findings,” Teachers CollegeRecord, vol. 107, no. 12, pp. 2543–2565, 2005.
[34] D. T. L. Shek, V. M. Y. Tang, and X. Y. Han, “Evaluation ofevaluation studies using qualitative research methods in thesocial work literature (1990–2003): evidence that constitutes awake-up call,” Research on Social Work Practice, vol. 15, no. 3,pp. 180–194, 2005.