MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index...
-
Upload
lea-delany -
Category
Documents
-
view
225 -
download
1
Transcript of MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index...
![Page 1: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/1.jpg)
MEQMEQAnalysisAnalysis
![Page 2: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/2.jpg)
OutlineOutline ValidityValidity ReliabilityReliability Difficulty IndexDifficulty Index Power of DiscriminationPower of Discrimination
![Page 3: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/3.jpg)
ValidityValidity““evidence present to support or evidence present to support or refute the meaning assigned to refute the meaning assigned to
assessment results”assessment results”
face validityface validitycontent validitycontent validitycriterion-related validitycriterion-related validityconstruct validityconstruct validity
![Page 4: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/4.jpg)
Face ValidityFace Validity high face validityhigh face validity
seemed valid (only)seemed valid (only) ~ person~ person
knee jerk & nervous systemknee jerk & nervous system for doctor => high face validityfor doctor => high face validity for lay people => low face for lay people => low face
validityvalidity MEQ => high face validityMEQ => high face validity
![Page 5: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/5.jpg)
Content ValidityContent Validity ~ sample & population~ sample & population MCQ => high content MCQ => high content
validityvalidity MEQ => low content MEQ => low content
validityvalidity
![Page 6: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/6.jpg)
ValidityValidity““evidence present to support or evidence present to support or refute the meaning assigned to refute the meaning assigned to
assessment results”assessment results”
face validityface validitycontent validitycontent validitycriterion-related validitycriterion-related validityconstruct validityconstruct validity
- don’t need the score- before using the test
- score needed
- after using the test
![Page 7: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/7.jpg)
Criterion-Related Criterion-Related ValidityValidity
Predictive validityPredictive validity MEQ score MEQ score && close observation close observation
score of real practice at ERscore of real practice at ER Concurrent ValidityConcurrent Validity
MEQ score MEQ score && VIVA score in the VIVA score in the same topicsame topic
Statistic = correlation Statistic = correlation coefficientcoefficient
![Page 8: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/8.jpg)
Construct ValidityConstruct Validity based on theorybased on theory
communication skill ~ leadershipcommunication skill ~ leadershipcorrelation OSCE : communication correlation OSCE : communication VS VS questionnaire : leadershipquestionnaire : leadership
good ethics ~ beloved doctorgood ethics ~ beloved doctorMEQ : medical ethics MEQ : medical ethics VSVS questionnaire : beloved doctor?questionnaire : beloved doctor?
![Page 9: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/9.jpg)
ReliabilityReliability stabilitystability internal consistencyinternal consistency equivalentequivalent
![Page 10: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/10.jpg)
StabilityStability - test retest reliability- test retest reliability parallel form reliability parallel form reliability - intra rater reliability- intra rater reliability
~ scoring key ~ scoring key statistics : correlation co statistics : correlation co
eff.eff. -01-01
![Page 11: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/11.jpg)
Internal Internal ConsistencyConsistency
[homogeneity][homogeneity] item - item correlationitem - item correlation item - total correlationitem - total correlation split half correlationsplit half correlation
![Page 12: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/12.jpg)
Item-Item Item-Item CorrelationCorrelation
each itemeach item DichotomousDichotomous
Phi CorrelationPhi Correlation IntervalInterval
Pearson’s Product Moment Pearson’s Product Moment CorrelationCorrelation
whole testwhole test Mean of ...Mean of ...
![Page 13: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/13.jpg)
Item-Total Item-Total CorrelationCorrelation
each item each item DichotomousDichotomous
Point Biserial Correlation Point Biserial Correlation IntervalInterval
Pearson’s Product Moment Correl Pearson’s Product Moment Correlationation
whole test whole test Mean of ... Mean of ...
![Page 14: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/14.jpg)
Spilt Half Spilt Half ReliabilityReliability
DichotomousDichotomous - 20 2Kuder Richardson (KR- 20 2Kuder Richardson (KR00
IntervalInterval - 21 2Kuder Richardson (KR- 21 2Kuder Richardson (KR1)1)
aaaaa aaaaaaaaaaa’ aaaaa aaaaaaaaaaa’
![Page 15: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/15.jpg)
EquivalentEquivalent parallel item on parallel item on
alternate form reliabilityalternate form reliability inter-rater reliabilityinter-rater reliability
agreementagreement kappakappa
![Page 16: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/16.jpg)
Difficulty IndexDifficulty Index [p][p] (mean H + mean L)(mean H + mean L)//2(full score)2(full score) p = 1 => very easyp = 1 => very easy p = 0 => very difficultp = 0 => very difficult expecting p = 0.2-0.8expecting p = 0.2-0.8
must => 0.7must => 0.7 should => 0.5should => 0.5
![Page 17: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/17.jpg)
Power of Power of DiscriminationDiscrimination
[r][r] (mean H - mean L)(mean H - mean L)//(full score)(full score) > 0.40 => very good item> 0.40 => very good item 0.30 - 0.39 => good item0.30 - 0.39 => good item 0.20 - 0.29 => borderline0.20 - 0.29 => borderline < 0.20 => poor item< 0.20 => poor item
![Page 18: MEQ Analysis. Outline Validity Validity Reliability Reliability Difficulty Index Difficulty Index Power of Discrimination Power of Discrimination.](https://reader033.fdocuments.us/reader033/viewer/2022052401/56649c8f5503460f94947e3f/html5/thumbnails/18.jpg)
ConclusionConclusion ValidityValidity ReliabilityReliability Difficulty IndexDifficulty Index Power of DiscriminationPower of Discrimination