Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.
-
Upload
jennifer-clayton -
Category
Documents
-
view
226 -
download
7
Transcript of Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.
![Page 1: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/1.jpg)
Reliability and Validity
Introduction to Study Skills & Research Methods (HL10040)
Dr James Betts
![Page 2: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/2.jpg)
Lecture Outline:•Definition of Terms
•Types of Validity
•Threats to Validity
•Types of Reliability
•Threats to Reliability
•Introduction to Measurement Error.
![Page 3: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/3.jpg)
Some definitions…• Validity
• Reliability
• Objectivity
![Page 4: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/4.jpg)
Types of Experimental Validity• Internal
• External
![Page 5: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/5.jpg)
Logical Statistical
Validity
![Page 6: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/6.jpg)
Logical Validity
![Page 7: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/7.jpg)
Statistical Validity
![Page 8: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/8.jpg)
Logical/Statistical Validity
![Page 9: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/9.jpg)
Interesting Example: Breast Cancer• Incidence: ~1 % (0.8 %)
(i.e. a positive result should be detected for approximately 1 in every 100 women tested)
• Sensitivity: ~90 % (87 %)(the mammogram is sensitive enough that approximately 90 in every 100 breast cancer patients will receive a positive result)
• Specificity: ~90 % (93 %)(the mammogram is specific enough that approximately 90 in every 100 healthy patients will receive a negative result).
Data from Kerlikowske et al. (1996)
![Page 10: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/10.jpg)
Quick Test
• What is the probability that a patient receiving a positive result actually has breast
cancer?
![Page 11: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/11.jpg)
Threats to Internal Validity
![Page 12: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/12.jpg)
e.g.
O1 T O2
PO3
RO4
Threats to Internal/External Validity
![Page 13: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/13.jpg)
Threats to Internal/External Validity
![Page 14: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/14.jpg)
Threats to External Validity
![Page 15: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/15.jpg)
Reliability
Subject 1 60 ml.kg-1.min-1 60 ml.kg-1.min-1 60 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 55 ml.kg-1.min-1 55 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 70 ml.kg-1.min-1 70 ml.kg-1.min-1
![Page 16: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/16.jpg)
Reliability
Subject 1 60 ml.kg-1.min-1 65 ml.kg-1.min-1 65 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 60 ml.kg-1.min-1 60 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 75 ml.kg-1.min-1 75 ml.kg-1.min-1
![Page 17: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/17.jpg)
Reliability
Subject 1 60 ml.kg-1.min-1 72 ml.kg-1.min-1 57 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 61 ml.kg-1.min-1 52 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 40 ml.kg-1.min-1 84 ml.kg-1.min-1
![Page 18: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/18.jpg)
Types of Reliability
Subject 1 60 ml.kg-1.min-1 63 ml.kg-1.min-1 57 ml.kg-1.min-1
Subject 2 55 ml.kg-1.min-1 56 ml.kg-1.min-1 48 ml.kg-1.min-1
Subject 3 70 ml.kg-1.min-1 65 ml.kg-1.min-1 66 ml.kg-1.min-1
![Page 19: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/19.jpg)
Rater Reliability (Objectivity)• Intrarater reliability
• Interrater reliability
![Page 20: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/20.jpg)
Threats to Reliability
![Page 21: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/21.jpg)
Measurement Errors• Ultimately, reliability is dependent on the
degree of measurement error in a given study
• The overall error in any measurement is comprised of both systematic and random error
• We will address measurement error further next week…
![Page 22: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/22.jpg)
Literature Search Assignment
• The handout lists 8 questions which can be answered through retrieving the corresponding source articles
• Answer as many as possible and hand them in at next week’s lecture
• Does not contribute to final assessment but will be marked
• DO NOT contact author or order articles.
![Page 23: Reliability and Validity Introduction to Study Skills & Research Methods (HL10040) Dr James Betts.](https://reader036.fdocuments.us/reader036/viewer/2022062307/5515e06455034638038b4bc6/html5/thumbnails/23.jpg)
Selected Reading• Atkinson, G. and A. M. Nevill. Statistical methods for
assessing measurement error (Reliability) in variables relevant to sports medicine. Sports Medicine. 26:217-238, 1998.
• Holmes, T. H. Ten categories of statistical errors: a guide for research in endocrinology and metabolism. American Journal of Physiology. 286: E495-501.
• Thomas J. R. & Nelson J. K. (2001) Research Methods in Physical Activity, 4th edition. Champaign, Illinois: Human Kinetics