“Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding...

58
“Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during the COVID-19 pandemic: More important now than ever Delivered by the FIP Pharmacy Practice Research Special Interest Group in Collaboration with Research in Social and Administrative Pharmacy and the Social and Administrative Pharmacy Section

Transcript of “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding...

Page 1: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

ldquoResponding to the Pandemic Togetherrdquo Programme

Episode 27 Evidence-based practice during the

COVID-19 pandemic More important now than

everDelivered by the FIP Pharmacy Practice Research Special Interest Group in Collaboration with

Research in Social and Administrative Pharmacy and the Social and Administrative Pharmacy

Section

Victoria Garcia Cardenas

bull Senior Lecturer University of Technology Sydney

bull Chair FIP Pharmacy Practice Research SIG

bull Associate Editor Research in Social and

Administrative Pharmacy

Email VictoriaGarciaCardenasutseduau

Moderator(s)

VGC_AF

I Provide relevant information and interim guidelines for pharmacists and the pharmacy workforce on

Coronavirus SARS-CoV-2COVID-19 pandemic

II Share and discuss strategies adopted by pharmacy leaders and workers - including our Member

Organisations ndash in response to the pandemic

III Describe sector or area-specific implications innovations and approaches adopted across pharmaceutical

science practice and education

IV Engage frontline workers of the health and pharmacy workforce to know about the realities facing them

around the world

V Discuss the implications of the pandemic on issues such as safety supply shortages that have been

exacerbated by COVID-19 across our nations and regions

VI Consider the impact of this disease on patients across age groups and with concurrent conditions

VII Assess and discuss the evidence behind treatments and the process of developing therapies vaccines and

tests

Welcome to the ldquoResponding to the Pandemic Togetherrdquo events

FIPrsquos Special Online Programme on COVID-19

These webinars aim to

To share ideas on webinar topics we should feature or if

yoursquod like to share your story on dealing with the pandemic

please email

linafiporg

Important Links amp Resources

FIP Covid-19 Information Hub

A comprehensive FIP webpage containing all of

our resources and outputs relating to COVID-19

including recordings of previous webinars

Link httpswwwfiporgcoronavirus

FIP Facebook Group ldquoCOVID-19 amp

pharmacyrdquo

Linkhttpswwwfacebookcomgroupscovid19and

pharmacy

Announcements

1 This webinar is being recorded and live streamed on Facebook

2 The recording will be freely available at wwwfiporgcoronavirus and

on our YouTube channel

3 You may ask questions by typing them into the QampA box

4 Your feedback is welcome (webinarsfiporg)

FIP Digital Events House Rules

copyFIP All the information in this video are confidential and cannot be copied

downloaded or reproduced without the formal approval of FIP (International

Pharmaceutical Federation)

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 2: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Victoria Garcia Cardenas

bull Senior Lecturer University of Technology Sydney

bull Chair FIP Pharmacy Practice Research SIG

bull Associate Editor Research in Social and

Administrative Pharmacy

Email VictoriaGarciaCardenasutseduau

Moderator(s)

VGC_AF

I Provide relevant information and interim guidelines for pharmacists and the pharmacy workforce on

Coronavirus SARS-CoV-2COVID-19 pandemic

II Share and discuss strategies adopted by pharmacy leaders and workers - including our Member

Organisations ndash in response to the pandemic

III Describe sector or area-specific implications innovations and approaches adopted across pharmaceutical

science practice and education

IV Engage frontline workers of the health and pharmacy workforce to know about the realities facing them

around the world

V Discuss the implications of the pandemic on issues such as safety supply shortages that have been

exacerbated by COVID-19 across our nations and regions

VI Consider the impact of this disease on patients across age groups and with concurrent conditions

VII Assess and discuss the evidence behind treatments and the process of developing therapies vaccines and

tests

Welcome to the ldquoResponding to the Pandemic Togetherrdquo events

FIPrsquos Special Online Programme on COVID-19

These webinars aim to

To share ideas on webinar topics we should feature or if

yoursquod like to share your story on dealing with the pandemic

please email

linafiporg

Important Links amp Resources

FIP Covid-19 Information Hub

A comprehensive FIP webpage containing all of

our resources and outputs relating to COVID-19

including recordings of previous webinars

Link httpswwwfiporgcoronavirus

FIP Facebook Group ldquoCOVID-19 amp

pharmacyrdquo

Linkhttpswwwfacebookcomgroupscovid19and

pharmacy

Announcements

1 This webinar is being recorded and live streamed on Facebook

2 The recording will be freely available at wwwfiporgcoronavirus and

on our YouTube channel

3 You may ask questions by typing them into the QampA box

4 Your feedback is welcome (webinarsfiporg)

FIP Digital Events House Rules

copyFIP All the information in this video are confidential and cannot be copied

downloaded or reproduced without the formal approval of FIP (International

Pharmaceutical Federation)

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 3: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

I Provide relevant information and interim guidelines for pharmacists and the pharmacy workforce on

Coronavirus SARS-CoV-2COVID-19 pandemic

II Share and discuss strategies adopted by pharmacy leaders and workers - including our Member

Organisations ndash in response to the pandemic

III Describe sector or area-specific implications innovations and approaches adopted across pharmaceutical

science practice and education

IV Engage frontline workers of the health and pharmacy workforce to know about the realities facing them

around the world

V Discuss the implications of the pandemic on issues such as safety supply shortages that have been

exacerbated by COVID-19 across our nations and regions

VI Consider the impact of this disease on patients across age groups and with concurrent conditions

VII Assess and discuss the evidence behind treatments and the process of developing therapies vaccines and

tests

Welcome to the ldquoResponding to the Pandemic Togetherrdquo events

FIPrsquos Special Online Programme on COVID-19

These webinars aim to

To share ideas on webinar topics we should feature or if

yoursquod like to share your story on dealing with the pandemic

please email

linafiporg

Important Links amp Resources

FIP Covid-19 Information Hub

A comprehensive FIP webpage containing all of

our resources and outputs relating to COVID-19

including recordings of previous webinars

Link httpswwwfiporgcoronavirus

FIP Facebook Group ldquoCOVID-19 amp

pharmacyrdquo

Linkhttpswwwfacebookcomgroupscovid19and

pharmacy

Announcements

1 This webinar is being recorded and live streamed on Facebook

2 The recording will be freely available at wwwfiporgcoronavirus and

on our YouTube channel

3 You may ask questions by typing them into the QampA box

4 Your feedback is welcome (webinarsfiporg)

FIP Digital Events House Rules

copyFIP All the information in this video are confidential and cannot be copied

downloaded or reproduced without the formal approval of FIP (International

Pharmaceutical Federation)

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 4: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Important Links amp Resources

FIP Covid-19 Information Hub

A comprehensive FIP webpage containing all of

our resources and outputs relating to COVID-19

including recordings of previous webinars

Link httpswwwfiporgcoronavirus

FIP Facebook Group ldquoCOVID-19 amp

pharmacyrdquo

Linkhttpswwwfacebookcomgroupscovid19and

pharmacy

Announcements

1 This webinar is being recorded and live streamed on Facebook

2 The recording will be freely available at wwwfiporgcoronavirus and

on our YouTube channel

3 You may ask questions by typing them into the QampA box

4 Your feedback is welcome (webinarsfiporg)

FIP Digital Events House Rules

copyFIP All the information in this video are confidential and cannot be copied

downloaded or reproduced without the formal approval of FIP (International

Pharmaceutical Federation)

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 5: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Announcements

1 This webinar is being recorded and live streamed on Facebook

2 The recording will be freely available at wwwfiporgcoronavirus and

on our YouTube channel

3 You may ask questions by typing them into the QampA box

4 Your feedback is welcome (webinarsfiporg)

FIP Digital Events House Rules

copyFIP All the information in this video are confidential and cannot be copied

downloaded or reproduced without the formal approval of FIP (International

Pharmaceutical Federation)

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 6: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Learning Objectives

bull To differentiate between the different levels of evidence

bull To identify the types of evidence that can be used to inform practice

bull To critically assess the published studies related to COVID19 and be able to

use them to inform practice where appropriate

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 7: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Evidence Based Practice

bull Evidence based practice requires that healthcare decisions are made based

on the best available current valid and relevant evidence and is essential

to deliver high quality patient care

bull Critical during the current COVID-19 pandemic

The concept and relevance of the webinarS

pea

ke

r 1 Levels of evidence

Sp

ea

ke

r 2 Critical appraisal of

evidence

Sp

ea

ke

r 3 Economic evidence

to inform decision making

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 8: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Speaker 1

Filipa Alves da Costa PhD

bull Public health Consultant WHO Regional Office for Europe

bull IUEM Associate Professor

bull FFUL Invited Professor

bull RON (National Oncology Register) Researcher in therapeutic effectiveness

bull Associate Editor of International Journal of Clinical Pharmacy

bull Chair Education Committee European Society of Clinical Pharmacy

bull iPACT Board

Email alvesdacostafgmailcom Filipa_A_C

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 9: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

What is evidence

bull Opinion evidence refers to evidence of what the witness thinks believes or infers in regard to

facts as distinguished from personal knowledge of the facts themselves In general witnesses

should testify only as to the facts observed and should not give opinion[1]

bull Evidence (noun) the available body of facts or information indicating whether a belief or

proposition is true or valid[2]

bull Evidence-based medicine the conscientious explicit and judicious use of current best

evidence in making decisions about the care of individual patients The practice of evidence-

based medicine means integrating individual clinical expertise with the best available external

clinical evidence from systematic research[3]

What are opinions

1 Wikipedia Available at httpsenwikipediaorgwikiOpinion_evidence 2 Dictionary

of Oxford Languages 3 Sackett DL Evidence-Based Medicine Seminars in

Perinatology 1997 21(1) 3-5

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 10: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Hierarchy of EvidenceHow to judge the quality of evidence provided by different studies

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 11: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Consensus Statements

ldquoExpert consensus statementrdquo implies review by recognized

organizations and widespread expert agreement

It should reflect a broad-based consensus representing more

than author opinions

It should not reflect the views of a few self-selected

individuals even if after conducting literature

Recommendations issued ought to be supported by existing

evidence the highest available to date Eg WHO

recommendations consider only systematic reviews there are

published studies with recommendations based on a single

published report of 10 cases

Foust AM Phillips GS Chu WC Daltro P Das KM Garcia-Pentildea P Kilborn T Winant AJ Lee EY

International Expert Consensus Statement on Chest Imaging in Pediatric COVID-19 Patient Management

Imaging Findings Imaging Study Reporting and Imaging Study Recommendations Radiology

Cardiothoracic Imaging 2020 2(2)

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 12: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Case study and Case Series

Case presentationhellipOn 11 February 2020 a 37-year-old man

presented to Wuhan Huo Shen Shan Hospital with

a history of fever dry cough and chest pain since

10 January 2020 The chest CT of this patient on

08 February showed multiple infiltrations in both

lungs consistent with viral infection But the RT-

PCR amplification of SARS-Cov-2 virus nucleic

acid from a nasopharyngeal swab was negative

He denied any other diseases before this onset

The initial physical examination revealed a body

temperature of 388 degC oxygen saturation (SPO2)

85ndash90 under ambient air respiratory rate of 40

breathsminute blood pressure of 14593 mmHg

and pulse of 119 bpm The laboratory results

reflected normal lymphocytes normal procalcitonin

(004 ngmL) and elevated C-reactive protein

(CRP 965 mgL) a-hydroxybutyrate

dehydrogenase (a-HBDH 318 IUL) and glutamyl

transpeptidase (GGT 136 IUL)hellip

1 Wang M Luo L Bu H amp Xia H Case Report One Case of Coronavirus Disease 2019 (COVID-19) in Patient Co-

Infected by HIV With a Low CD4+ T Cell Count International Journal of Infectious Diseases 2020 2 Makurumidze R

Coronavirus-19 Disease (COVID-19) a case series of early suspects reported and the implications towards the response

to the pandemic in Zimbabwe Journal of Microbiology Immunology and Infection 2020

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 13: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Cross-sectional studies

bull Measure the prevalence of

conditions or characteristics of

people in a population at a point in

time or over a short period

bull Classified as descriptive studies for

large populations but can also

explore risk factors associated with

particular illness or behaviour

bull Useful for planning public health

interventions

1 Online survey of 4850 Malaysian residents 13

knowledge items 3 on attitudes and 3 on practices gt80

taking precautions to avoid crowds hand hygiene face

masks by 51

2 Online self-reported survey from 3388 people from South

Arabia Older adults are likely to have better knowledge

and practices than younger people (pgt0001)

3 UK bathers were more likely to report skin ailments (

AOR=264 95CI 123 to 565 ear ailments

(AOR=377 95CI 184-773 and any symptoms of

illness (AOR=373 95CI 263-529

General characteristics Some examples

1 Azlan et al Public knowledge attitudes and practices towards COVID-19 A cross-sectional study in Malaysia PLoS ONE 2020 15(5) 2

Al-Hanawi et al Knowledge Attitude and Practice Toward COVID-19 Among the Public in the Kingdom of Saudi Arabia A Cross-Sectional

Study Frontiers in Public Health 20208217 3 Leonard et al A cross-sectional study on the prevalence of illness in coastal bathers

compared to non-bathers in England and Wales Findings from the Beach User Health Survey Water Research 2020 176(1)

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 14: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Case-control studies

Unhealthy

Healthy

Who was exposed

Odds Ratio

Hernaacutendez-Garduntildeo E Obesity is the comorbidity more strongly associated for Covid-

19 in Mexico A case-control study Obesity Research amp Clinical Practice 2020

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 15: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Cohort studies

ExposedUnexposed

How long will be monitoring

us

Roughly 20 years

Researcher

UnexposedExposed20 years later

Time for the count

Relative Risk

Hazard Function

Xiao et al Comparison of Hospitalized Patients With ARDS Caused by

COVID-19 and H1N1 Chest 2020 158(1)195-205

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 16: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Clinical Trials

Relative Risk

Hazard Function

News releases from National Institute of Allergy and Infectious Diseases (NIAID) Phase

3 clinical trial of investigational vaccine for COVID-19 begins Multi-site trial to test

candidate developed by Moderna and NIH

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 17: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Clinical Trials

httpsclinicaltrialsgovct2who_table

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 18: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Systematic Reviews

bull Research question operationalized using PICOTS

bull Intervention must be clearly defined

bull Outcomes standardised (eventually divided into

primary vs secondary)

bull Searches made in ge 3 databases

bull Study designs should ideally be identical

(sometimes not feasible

bull Extracted studies analysed and appraised for

quality and risk of bias

bull Results may be synthesized narratively and in

tabular form

bull Ovid MEDLINE Embase CINAHL and the WHO Global Index

Medicus

bull ldquoAs randomization of quarantine is unethical and not feasible for

the diseases in question we considered non-randomized

studies of interventions to be the best potentially available

empirical evidencehellip we also included modelling studies

because we did not yet expect empirical studies to be

availablerdquo Cohort studies Case-control studies time series

Interrupted time series Case series Mathematical modelling

studies

General characteristics One example (rapid review)

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 19: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Systematic Reviews

Nussbaumer-Streit et al Quarantine alone or in combination with other public health measures to control

COVID-19 a rapid review Cochrane Database of Systematic Reviews 2020 Issue 4 Art No

CD013574

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 20: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 21: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Meta-analysis

Chu et al Physical distancing face masks and eye protection to prevent person-to-person transmission

of SARS-CoV-2 and COVID-19 a systematic review and meta-analysis Lancet 2020 3951973ndash87

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 22: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

All studies have their place as long as well conducted

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 23: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Speaker 2

Fernanda Stumpf Tonin PhD

bull HEOR Consultant

bull Researcher Federal University of Paranaacute Brazil

bull Vice-chair FIP Pharmacy Practice Research SIG

bull Member of the Editorial Board Pharmacy Practice

Email ffstoningmailcom StumpfTonin

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 24: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Critical appraisal of evidence

Suboptimal research

27 of publications are redundant

20 have methodological flaws

20 are unpublished

17 are decent but not useful

13 misleading conclusions

3 have a scientificclinical meaning

To effectively practice as an evidence-based practice provider

Ioannidis JP The Milbank quarterly 201694(3)485-514

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 25: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Critical appraisal of evidenceCOVID-19 era increasing value reducing waste

bull To know where to find

information

bull To be able to identify select

and appraise the best and

most up-to-date evidence

bull To integrate these findings

with your own clinical

experience and patients

values

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 26: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Critical appraisal of evidenceCOVID-19 era where to find evidence

httpscovid-evidenceorg

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 27: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Purpose of critical appraisal

bull Critical appraisal process of systematically assessing the outcome of scientific research (evidence) to judge its

trustworthiness value and relevance in each scenario

bull Aims to evaluate the level and quality of evidence to support decision-making

How certain are we about the results (validity)

How applicable are the results to practice (applicability translational capacity)

bull Critical appraisal is essential to

- Combat information overload

- Identify papers that are clinically relevant

- Continuing professional development

Supporting decision-making

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Carrying out critical appraisal ndash basic steps

Carefully read the study

Define study design ndash evaluate research methods

Check minimum standards conductionreporting (checklists)

Address quality validity of results and compare to other studies

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 28: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Enhancing the QUAlity and Transparency Of health Research

httpwwwequator-networkorg

Conducting and reporting studiesOverall recommendations

The EQUATOR Network

bull International initiative

bull Improve the reliability and

value of published health

research literature

bull Transparent and accurate

reporting

bull Wider use of robust

reporting guidelines

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 29: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Critical appraisal of evidence

Some initial appraisal questions include

1 Is the evidence from a known reputable source

2 Has the evidence been evaluated in any way If so how and by whom

3 How up-to-date is the evidence

4 Were all important outcomes considered How were they measured

5 Is that a reliable way to measure

6 How large was the effect size

7 What implications does the study have for your practice Is it relevant

8 Can the results be applied into practice (benefit-risk ratio)

9 Are the benefits worth the costs and potential risks

Basic steps

Sackett DL et al Evidence based medicine what it is and what it isnrsquot BMJ 1996312(7023)71ndash72

MacInnes A Lamont T Critical appraisal of a research paper Scott Uni Med J 20143(1)10ndash17

Burls A What is critical appraisal London Hayward Medical Communications 2016

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 30: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Systematic review and meta-analysisCOVID-19 evidence

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 31: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Systematic review and meta-analysisCOVID-19 evidence

Effects of four types of integrated Chinese and Western

medicines for the treatment of COVID-19 in China a

network meta-analysisRev Assoc Med Bras (1992) 2020 Jun66(6)771-777

Network plot multiple

comparisons of interventions

Ranking

analysis

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 32: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

GRADE

bull Provides a transparent and structured approach to making judgments about the certainty of the evidence

bull Offers a transparent process to making recommendations and decisions

bull Currently used by over 100 organizations globally including the World Health Organization

bull Ideally applied to rate the certainty of a body of evidence in a well-conducted and up-to-date evidence

synthesis (eg setting population intervention comparator outcomes) with summary tables

bull Although appropriately sophisticated in its full execution it can answer questions and be relayed to decision-

makers by breaking its components down into straightforward questions about

- the certainty of evidence

- the criteria for making decisions or recommendations

Grading of Recommendations Assessment Development and Evaluation

wwwgradeworkinggrouporg

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 33: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

GRADEGrading of Recommendations Assessment Development and Evaluation

bull Study Design

bull Quality

bull Inconsistency

bull Indirectness

bull Imprecision

bull Other factors

wwwgradeworkinggrouporg

Guide

recommendations

StrongWeak

FavorsAgainst

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 34: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Schuumlnemann HJ et al J Clin Epidemiol 2020 Jun 5S0895-4356(20)30425-X

bull In situations of emergencies and urgencies such

as the COVID-19 pandemic GRADE can similarly

be used to express and convey certainty in

intervention effects test accuracy risk and

prognostic factors consequences of public health

measures and qualitative bodies of evidence

bull Requirements for emergency urgency rapid and

routine GRADE assessment may differ but should

transition from one to another

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 35: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Implications amp Take-home messages

bull We should get used to always evaluate the provenance and quality of information

bull Critical appraisal looks at the way a study is conducted and evaluates factors such as internal

validity generalizability and relevance

bull Evidence and recommendations generation need high quality studies (data confidence)

bull Decisions related to patient value and care are carefully made following an essential process of

integration of the best existing evidence clinical experience and patient preference

bull GRADEing the certainty of the available evidence is more important than ever because of the

unprecedented pressure for action and the large number of people affected by decisions

To effectively practice as an evidence-based practice provider

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 36: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Speaker 3

Dalia Dawoud PhD

Associate Editor

Pharmacoeconomics and Outcomes Research

Research in Social and Administrative Pharmacy (RSAP)

and Value in Health Elsevier UK

amp

Associate Professor

Faculty of Pharmacy Cairo University Egypt

Email ddawoudhotmailcom drddawoud

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 37: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

bull No health care system in the world can

provide every effective intervention

Resources are limited and wants are

limitless (Scarcity)

bull If you provide more of one service you

will have to provide less of another

(Opportunity cost)

bull Choices and trade-offs must be made

Why

httpsantiwarwarvetcomflattening-the-curve

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 38: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

Health economics utilises economic analysis methods to inform decision

making regarding the allocation of the scarce resources available by identifying

interventions that most likely to provide the best value for every pound$euro spent (ie

cost-effective)

How

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 39: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

Economic Evaluation

ldquoThe comparative analysis of alternative courses of action in terms

of both their costs and consequencesrdquo (Drummond et al 2015)

bull The type of an economic evaluation is largely determined by

bull The nature and measure of the outcomes considered

bull The presence of evidence (or assumptions made) regarding (non-)

equivalence of outcomes

bull How the analysis results are presented

How

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 40: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

bull Cost-consequences analysis (CCA)

bull Includes all outcomes

bull Reports costs and outcomes separately

bull Cost-effectiveness analysis (CEA)

bull Focuses on one primary outcome

bull Disease specific expressed in natural

units (eg number of strokes avoided)

bull Cost-utility analysis (CUA)

bull Focuses on one primary outcome

bull Generis outcome eg Quality Adjusted

Life Years (QALYs) or Disability

Adjusted Life Years (DALYs)

bull Cost-benefit analysis (CBA)

bull Measures both benefits and costs in

monetary terms

How

Economic Evidence The Missing Piece

NO YES

Examines only

consequences

Examines only

costs

1A PARTIAL EVALUATION 1B

bull Outcome

description

bull Cost

description

3A PARTIAL EVALUATION 3B

bull Efficacy or

effectiveness

evaluation

bull Cost analysis

bull Cost-

minimisation

analysis

2 PARTIAL EVALUATION

bull Cost-outcome description

4 FULL ECONOMIC EVALUATION

bull Cost consequences analysis

bull Cost-effectiveness analysis

bull Cost-utility analysis

bull Cost-benefit analysisN

OY

ES

1 Are both costs (inputs) and consequences (outputs) examined

2 A

re a

t le

ast 2 a

ltern

atives c

om

pare

d

Drummond et al 2015

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 41: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

bull Economic Evaluation is most useful after the following

bull Efficacy studies which aim to answer the question ldquoCan the intervention

work

bull Safety studies which aim to answer the question ldquoDoes it do more good

than harmrdquo

bull Effectiveness studies which answer the question ldquoDoes the intervention

work when appliedrdquo

The bottom line is that if an intervention is not effective it is not cost-effective

When

Economic Evidence The Missing Piece

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 42: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

A Alongside a clinical study

Collecting data on both costs and

consequences simultaneously from a

single study (mostly phase III RCT)

Approaches

Economic Evidence The Missing Piece

B Using Economic Modelling

Mathematical simulation of the costs and

consequences attached to using each alternative

using data from various sources(eg Systematic

reviews and meta-analysis epidemiological studies

RCTs observational studies)

The following are broadly the main steps of conducting a full economic evaluation

1 Identifying measuring and valuing outcomes

2 Identifying measuring and valuing costs

3 Combining costs and outcomes

4 Assessing uncertainty and drawing conclusions to inform decision-making

5 Optional Assessing Value of Information to inform future research investment

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 43: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

bull Critical appraisal of published economic evaluation studies allows us to assess the methodological quality and applicability of these studies and their results to current clinical practice

bull The Critical Appraisal Skills Program (CASP) proposed a simple checklist to appraise published economic evaluations in terms of quality usefulness and applicability

bull This checklist prompts the reviewer to answer the following

questions

bull Is the economic evaluation valid

bull How were costs and consequences assessed and compared

bull Will the results help in purchasing services for local people

Critical Appraisal

1CASP Checklist for economic evaluation studieshttpmediawixcomugddded87_3b2bd5743feb4b1aaac6ebdd68771d3fpdf

This Photo by Unknown Author is licensed under CC BY-SA-NC

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 44: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

bull Guidelines for conducting economic evaluations also exist to provide a set of methodological standards that should be followed

bull These guidelines are usually proposed by the decision makers who are going to use the results of these studies in their decision making to ensure applicability of the results to their jurisdictions

bull An example of these guidelines is the ldquoGuide to the Methods of Technology Appraisalrdquo published by NICE in April 20131

Economic Evidence The Missing PieceCritical Appraisal

1 NICE Guide to the methods of technology appraisal 2013 httpswwwniceorgukarticlepmg9chapterthe-reference-case

httpstoolsispororgpeguidelines

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 45: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

bull The British Medical Journal Value in Health RSAP and other peer-reviewed journals publishing economic evaluations adopted a 24 item ldquochecklistrdquo for reporting of economic evaluations developed by ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices Task Force

Reporting Standards

Economic Evidence The Missing Piece

1 Husereau D et al (2013) ISPOR TASK FORCE REPORT Consolidated Health Economic Evaluation Reporting Standards

(CHEERS)mdashExplanation and Elaboration A Report of the ISPOR Health Economic Evaluation Publication Guidelines Good

Reporting Practices Task Force Value in Health httpswwwvalueinhealthjournalcomarticleS1098-3015(13)00022-3pdf

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 46: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

bull No economic evaluation of COVID-19 related

interventions or strategies published so far

bull One report from USA ICER used economic modeling

to establish the value-based price benchmark of

remdesivir using economic evaluation (CUA)

bull But a number identified in the literature focused on a

large number of mitigation strategies used in

previous outbreaks such as H1N1

Examples

httpsicer-revieworg

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 47: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Economic Evidence The Missing Piece

bull Screening

bull Disease surveillance

networks

bull Contact tracing

bull Face masks

bull Hand washing

bull Social distancing

bull Self-isolation

bull Antiviral prophylaxis

bull Antiviral treatment

bull Antiviral stockpiling

bull Vaccination

bull Border control

bull School closure

Examples

ldquothis study estimates that the use of facemasks by 10 25 and 50 of the population could reduce economic losses by $478 billion $570 billion and $573 billion respectivelyrdquo

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 48: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

The Times

Daily Express

Birmingham medical news

The Guardian

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 49: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Putting evidence into action

Evidence-based Practice

bull Teaching clinicians how to

find the evidence to answer

clinical questions

bull Individual clinicians

bull Bottom-up approach

The role of clinical guidance

Clinical Guidelines and HTA

bull Advising clinicians how to practice based on evidence

bull Health systems

bull Top-down approach

The open university Photo John WildgooseGetty Images

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 50: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Putting evidence into action

ldquoStatements that include recommendations intended to optimize patient outcomes that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care optionsrdquo1

Clinical (Practice) Guidelines

1 Committee on Standards for Developing Trustworthy CPGs (IOM-AHRQ 2011)

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 51: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Putting Evidence Into ActionRapid Guidelines

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 52: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Health Technology Assessment (HTA)

Putting evidence into action

ldquoA multidisciplinary process that uses explicit methods to

determine the value of a health technology at different points in its

lifecycle The purpose is to inform decision-making in order to

promote an equitable efficient and high-quality health systemrdquo OrsquoRourke et al 2020

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 53: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Decision-making under uncertainty and an evolving evidence-base

Uncertainty

Putting Evidence Into Action

Rutter et al Managing uncertainty in the covid-19 era BMJ Opinion July 2020 httpsblogsbmjcombmj20200722managing-uncertainty-

in-the-covid-19-era

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 54: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Clinical judgment

bull ldquoGuidelines not tramlinesrdquo Sir David Haslam

Putting Evidence Into Action

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 55: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Thank You

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 56: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Question Time

copyFIP All the information in this video are confidential and cannot be copied downloaded or reproduced

without the formal approval of FIP (International Pharmaceutical Federation)

Please use the chat board to log your questions amp comments

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event

Page 57: “Responding to the Pandemic Together” Programme Episode 27: … · 2 days ago · “Responding to the Pandemic Together” Programme Episode 27: Evidence-based practice during

Thank you for participating

Please provide your feedback through the 4-question

survey that will appear to you at the end of the event