Teaching Registrars Research Methods

Variable definition and quality control of measurements

Prof. Rodney Ehrlich

Learning objectives

1. Scales of measurement

2. Conceptual vs operational variables

3. Precision, accuracy and validity of measurements:• Understand• Maximise• Assess

Measurement scales

Categorical:

• Nominal (no natural order), e.g. blood group• Ordinal, e.g. cancer staging

Continuous:

• Discrete (counts), e.g. outpatient attendance• “True” continuous, e.g. haemoglobin

Defining your variables

Conceptual variable = everyday term, or alternatively, theoretical construct

Operational variable = what is actually measured

Defining your variables: examples

• Renal function • Alcoholism• Risk taking behaviour• Chronic pain• Obesity

Quality control of measurement

“Measuring instrument” = questionnaire, laboratory test, clinical judgement.

Precision = reliability, repeatability or reproducibility

Accuracy = proximity to true value

Validity = subset of accuracy

Precision

Repetition between occasions, testers, instruments, gives same result.

• Lack of reliability may also indicate an accuracy or validity problem but the two are separable, at least in theory.

• Precision is not determinable in a single measurement.

2nd

Measure-ment

High Normal Total

High 12 2

Normal 8 178

200

Example (dichotomous variable): precision of BP measurement

First measurement.

What is the precision (reliability) of the BP measurement?

Measuring precision

Categorical

Percent agreement (concordance); Kappa statistic (takes chance agreement into

account)

Continuous

Various (see Hulley, Ch. 12) (NOT correlation coefficients)

Accuracy

Measurement agrees with another measurement accepted to be the truth, the so-called “gold standard”. Most intuitive for physical and physiological measurements

Validity

Where variable being measured is abstract, subjective, complex, etc., where gold standard debatable or not available

Types of validity

• Face validityMeasurement, or question, makes sense to you, interviewers, experts, subjects, et al.

• Construct validityMeasurement agrees with other operational measurements of the same concept.

Example: depression

• Criterion validity Measurement agrees with a “gold standard”.

Measuring criterion validity

Categorical

Sensitivity: proportion of true positives testing positive on the instrument

Specificity: proportion of true negatives testing negative on the instrument

Continuous

More complex, but often involves choosing cutpoints, i.e. categorising as positive/negative

Has your

child ever had chicken pox?

Yes No Total

Yes 60 20

No 90 30

200

Example of criterion validity of recall (categorical)

Chicken pox antibodies in blood?

Sensitivity, specificity;Implications?

Lack of precision/reliability = “random error”

Descriptive study e.g. prevalence of hypertension?

Wider confidence interval (reduced power); Need greater sample size (or repeated

measurements) for same power

Comparative study e.g. does nurse home visiting improve hypertension control?

Same as for descriptive

Lack of accuracy/validity= “systematic error”

Descriptive study Biased estimate; Cannot remove by increasing sample size.

Comparative study If affects both groups equally, will mask a true

difference or association; If affects the two groups differently, could mask

true difference or create a spurious difference or association (e.g. recall bias).