The COSMIN studyCOnsensus-based Standards for the selection of health status Measurement INstruments

Wieneke Mokkink, Caroline Terwee, Paul Stratford, Jordi Alonso, Donald Patrick, Dirk Knol, Lex Bouter, Riekie de Vet

Outline presentation

• Background– Methodological guidelines

• COSMIN Delphi Study– Purpose– Methods– Preliminary results

Why methodological guidelines?

Importance of high-quality studies- Detection of potential bias that may influence the results

- Better and more realistic estimates of results- Greater acceptance of results in health care community

Moher et al. Controlled Clinical Trials 1995;16:62-73

Accepted guidelines

• RCT: Delphi List (Verhagen et al.)• Diagnostic research: QUADAS

Why a new guideline?

• Several checklists for measurement properties exist (SAC MOS/Lohr, Bombardier & Tugwell, Andresen, Terwee)

• No consensus on terminilogy and defintions• None is generally accepted and widely used• Most chance of getting enough support is by

developing a checklist in an international consensus study, i.e. a Delphi Study

Aim of the COSMIN study

1. Which measurements properties should be included in the assessment of evaluative HR-PROs, and how should they be defined?

2. How should these measurement properties be assessed in terms of study design and statistical analysis? (i.e. standards)

Available as a checklist

Methods (1)

• International Delphi study• Steering committee• Background panel members:

– clinical medicine– biostatistics– psychology (psychometrics)– epidemiology (clinimetrics)

Methods (2)

Design delphi rounds:• Four written rounds

– which measurement properties (term and definitions)– how selected properties should be assessed (standards)– achieve consensus on complete checklist

• 5 point scale + arguments and comments– strongly disagree/disagree/no opinion/agree/strongly agree

• Consensus: ≥67% (strongly) agreed

Focus of COSMIN

HR-PRO instruments used in an evaluative application• PRO: endpoint derived from patient report• HR: not adherence or satisfaction with care

Flow chart

Invited: N=91

Agreed to participate: N=57

Flow chart

Invited: N=91

Agreed to participate: N=57

Round 1: N=57

Round 2: N=54

Round 3: N=52

Round 4: N=51

Responses: N=42 (74%)

Responses: N=31 (58%)

Responses: N=25 (48%)

Responses: N=27 (53%)

Drop out: N=3

Drop out: N=2

Drop out: N=1

Products of COSMIN

• Taxonomy (consensus on terms and definitions)• COSMIN checklist

Reliability

Content validity Construct validity Criterion validity

face validity

concurrent validity

predictive validity

cross-cultural validity†

hypotheses testing,e.g. convergent validity,

divergent validity, known group validity

structural validity*

* necessary for multi-item instruments, † necessary for cross-cultural comparison

Validity

Responsiveness

Results (2) Structure COSMIN checklist

3 Sections:- Descriptive information about the HR-PRO

instrument- Descriptive information about a study of a

measurement property- Evaluation of methodological quality of studies of

measurement properties, i.e. standards

Section 1

Descriptive information of the HR-PRO instrument• Development process• Format• Translation process• Interpretability

Example (1)

Section 2

Descriptive information of the study • IRT study

Example (2)

Section 3

Evaluation of methodological quality of the studies

• Term• Definition• Standard

– design requirement– statistical analysis

Standard for internal consistency

Future

• Selecting an HR-PRO instrument– Content– Practical issues– Measurement properties

• Standards

• Criteria of adequacy for good measurement properties

• Inter-rater reliability study of COSMIN checklist

Many thanks to …

Neil Aaronson Linda Abetz Elena Andresen Dorcas Beaton Martijn Berger Giorgio Bertolotti Monika Bullinger David Cella Joost Dekker Dominique Dubois Anne Evers Diane Fairclough David Feeny Raymond Fitzpatrick Andrew Garratt Francis Guillemin Dennis Hart Graeme Hawthorne Ron Hays Elizabeth Juniper Robert Kane Donna Lamping

Marissa Lassere Matthew Liang Kathleen LohrPatrick Marquis Chris McCarthy Elaine McColl Ian McDowell Don Mellenbergh Mauro Niero Geoffrey Norman Manoj Pandey Luis Rajmil Bryce Reeve Dennis Revicki Margaret Rothman Mirjam Sprangers David Streiner Gerold Stucki Giulio Vidotto Sharon Wood-Dauphinee Albert Wu