World Conference Nursing Dubai 2015
-
Upload
dr-thomas-kropmans -
Category
Healthcare
-
view
267 -
download
0
Transcript of World Conference Nursing Dubai 2015
![Page 1: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/1.jpg)
“Back to the future: An online OSCE management Information system for nursing OSCEs"
Dr Thomas JB Kropmans
![Page 2: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/2.jpg)
![Page 3: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/3.jpg)
Back to the future, the movie
![Page 4: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/4.jpg)
Developed by Ronald Harden in 1975 Medicine, nursing, dentistry, veterinary medicine, law, health sciences Student competency being assessed in consecutive series of stations Paper burden
Objective Structured Clinical Examinations e.g. OSCE
![Page 5: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/5.jpg)
Admission interviews (Students) Selection interviews (HR) Objective Structured Clinical Examinations (Medicine, Nursing, Health Care, Law) Workplace Based Assessment
Where to be used
![Page 6: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/6.jpg)
100 students x 15 stations = 1500 paper forms Not cost effective A 6-station OSCE report 327.5 hours of staff and faculty
time for each rotation of students (Ratanawongsa et al., 2008; Walsh & Smith, 2006)
US$ 6.90, equivalent to €4.70 per student per station (Cusimano et al., 1994)
We administer 11, 7-12 station OSCEs for a cohort of 670 students, produces 9380 assessment forms. To produce final OSCE results, the administrative cost of this procedure is €29,500, which is €2.80 per paper form.
Due to high error rates 30% (unpublished data)
Disadvantages
![Page 7: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/7.jpg)
Only student ability to pass an examination is being assessed Difficulty of the assessment is not taken into account Heterogeneity of measurement instruments (stations) limits comparability Comparison of student competence across institutions and assessment (OSCE) settings is required Universal adoption of a standardized instrument is recommended (Clinical Teacher September 2014)
Without standard setting
![Page 8: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/8.jpg)
Objective Structured Clinical Examinations e.g. OSCE
![Page 9: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/9.jpg)
Of 1000 forms 300 contains errors Low Cronbach’s Alpha (< 0.8 poor) Pass mark of 50%? Introduction Online Marking Tool in December 2008
30% based upon error
![Page 10: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/10.jpg)
Assess clinical skills or non-cognitive skills Determine between ‘good’ and ‘bad’ performance To provide adequate and timely feedback Reliable decision on ‘pass’ or ‘fail’
Purpose of a (formative or summative) clinical skills exam?
![Page 11: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/11.jpg)
Real time solution
![Page 12: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/12.jpg)
Real Time Data
![Page 13: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/13.jpg)
![Page 14: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/14.jpg)
Cronbach’s Alpha Internal consistency
![Page 15: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/15.jpg)
Issues with examiners
![Page 16: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/16.jpg)
Skew distribution • Median = 74%; min score = 4 (20%) and max 20 (100%)
Distribution of scores
74% 4
100%
Nr o
f stu
dent
s
20 20%
50%
![Page 17: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/17.jpg)
Professional impression examiners
![Page 18: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/18.jpg)
0.0
1.0
2.0
3.0
4.0
5.0
6.0
7.0
8.0
9.0
10.0
0 1 2 3 4
Stat
ion
scor
es
Method 1: Global score (Fail=0, Borderline=1, Clear Pass=2, Good Pass=3, Excellent=4)
Method 1 Single borderline score
Borderline Regression Analysis
![Page 19: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/19.jpg)
Improved pass/fail decisions
![Page 20: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/20.jpg)
User Acceptance Test n=18
![Page 21: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/21.jpg)
Satisfaction using the E-OSCE package 88.40% (min- 74; max = 100)
![Page 22: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/22.jpg)
22 prestigious universities & professional bodies >200 OSCEs successfully administered Student and Examiner results being analysed Cost reduction of 70% Error reduction by 30%
19% MORE NUIG students fail after introducing BRA Internal consistency varies from 0.45 – 0.85 Cut-off score varies between stations from 40 – 75% Generalisability Kappa between departments 0.4 – 0.9
![Page 23: World Conference Nursing Dubai 2015](https://reader033.fdocuments.us/reader033/viewer/2022052514/58f1cbf21a28ab8a558b457d/html5/thumbnails/23.jpg)
References