Evaluation of Diagnostic Tests Presenter: Akash Ranjan Moderator: Dr Chetna Maliye.
E CONOMIC E VALUATION Moderator: Dr. Chetna Maliye Presenter: Rohan.
-
Upload
melvin-simpson -
Category
Documents
-
view
215 -
download
0
Transcript of E CONOMIC E VALUATION Moderator: Dr. Chetna Maliye Presenter: Rohan.
ECONOMIC EVALUATION
Moderator: Dr. Chetna Maliye
Presenter: Rohan
FRAMEWORK Definition The need of Evaluation Responsibility for Evaluation Process of Evaluation Requisites to Programme Evaluation Sequential Steps Methods of Evaluation Assessment of Components Types of Evaluation Programme Evaluation
EVALUATION Definition:
The comparative analysis of alternative courses of action in terms of both their costs and consequences' (Drummond, et al., 1997).
It is the element-by-element examination of the expenditure to determine how resources have been spent.
A critical process needed to solve management problems
Helps to understand how funds have been used
Guides in optimal utilization of scarce resources
USES OF ECONOMIC EVALUATION:
Accountability Assessing Efficiency Assessing Equity Assessing priorities Making cost projections Considering cost Recovery
EVALUATION PROVIDE BASIS FOR:
Determinant the extent of attainment of objectives and progress towards goal
Establishing revised standards of performance Identifying strong and weak points/ dimensions/ aspects
of programme Challenging, revising/ modifying programme assumptions
by testing the implicit rationale of the programmatic intervention
Suggesting changes in program operations and objectives Clarifying programme by requiring operational definitions
in measurable terms Identifying inconsistencies between programme
objectives and activities.
EVALUATION PROVIDE BASIS FOR:
Justifying the programme to funding sources and the public
Testing alternative approaches and procedures for service delivery
Determining the coverage and availability of the programme relating to the existing community needs
Ensuring the provision of services which satisfy existing standards
Determining the resources required to respond to community needs both in terms of making service available and desired impacts of services provided
Comparing the allocation of project resource allocation decisions
Identifying the possible side effects of the programme
ECONOMICAL AND FINANCIAL COST ANALYSIS
Variable Financial analysis Economic analysis
Costs Monetary cost. Economic (“opportunity”) cost.
e.g, actual monetary payments for human resources, materials, or infrastructure
e.g, volunteer workers and donated goods
Valuation using market interest rates.
Valuation using a discount rate
Outcomes Income; expenditure; cash flow; profit; end-of-period balance; internal financial rate of return; net present financial value.
Benefit–cost ratio; internal economic rate of return; net present value.
Financial consequences Resource consequences
EVALUATION:
Evaluation could be divided into Total Partial Time Related
Periodicity: Progress and efficiency:-Once a year Effectiveness: In longer time than a year Impact: Longer time at least five years from
the inception of the programme
WHO CAN USE COST ANALYSIS
Government departments such as ministries NGO’s which largely exists to promote the social
good,. Individual or Institutional decision makers such as
hospitals. Private enterprises which produce goods where
there is demand and potential for profit. Individual and consumers who make purchase
decisions based on their perceptions of needs External donors which may include bilateral
government agencies, multilateral agencies and international agencies
REQUISITES TO PROGRAMME EVALUATION:
Criteria reasonable to employ before evaluating any Programme:-
Programme which have the potential to make an impact on a large number of people.
Programme which have a higher potential fro inflicting harm on people.
The amount and type of resources required by a specific programme.
Pilot or demonstration projects merit a relatively high priority.
Following Programme should not be evaluated:
Programme which are too new or too disorganized.
Resources for evaluation are inadequate The key administrators are not committed to it or
its use.
STEPS FOR COST-ANALYSIS
Research Question
Plan Ecnomic Evaluation study
•Setting scope of study
•Deciding on time trend
•Selecting sample
•Drawing iternary & checklist
Cost estimation Identification and choice of costs Quantification of costs Valuation of costs Adjustment for differential timing
Appropriate cost analysis methods
Analysis, Interpretation and Presentation
Assessment of Impact of uncertainty
Impact estimation Identification and choice of impacts Quantification of impacts Valuation of impacts Adjustment for differential timing
METHOD OF EVALUATION:
Components Assessments of the components and Indicators.
INDICATORS:
Output Indicators: Measure the accomplishments of the programme objectives.
Process Indicators: Measure the manner and extent of carrying out the processes under consideration
Product Indicator: Measure the outcome of this process
Progress Indicators:
Health Policy Indicators : like allocation of resources, community involvement
Health status Indicator: like weight of new born, wt/age ratio in children.
Social and economic indicators like population growth , GNP, GDP
Provision of health care : Availability, Accessibility, utilization of services.
Indicators of coverage: Availability of safe water, sanitary facilities, MCH
Identification of main costs and their sources
Quantification of costs
Valuation of costs and discounting.
COST- ESTIMATION
IDENTIFICATION OF COSTS AND THEIR SOURCES
InterventionIntervention
Direct CostDirect Cost Indirect CostsIndirect Costs
Wider cost implications to society eg. lost production.Time receiving medical care
Wider cost implications to society eg. lost production.Time receiving medical care
Non-health services Eg. Patient transportation, AdministrationResearch & Development
Non-health services Eg. Patient transportation, AdministrationResearch & Development
Health services resource use. Eg. Inpatient, outpatient, tests, drugs
Health services resource use. Eg. Inpatient, outpatient, tests, drugs
Costs to family and friends.
Costs to family and friends.
Perspective is important Range of costs justified by perspective
SOURCES OF COST DATA Routine Information Survey :
◦ Market prices & Labor statistics
Periodic information sources- ◦ contracted company, ◦ government ministry or◦ NGO
Specialist surveys or studies ◦ Household surveys ◦ scientific studies
Expert opinion
QUANTIFICATION OF COST DATA Need to quantify resource use in appropriate physical and
natural units◦ hours, days, miles etc
Direct costs are mostly assessed, and categorised as:◦ Capital costs (buildings, equipment)◦ Overheads (jointly used resources, such as heating and
lighting, administration and catering)◦ Labour (medical and non-medical staff)◦ Consumables (disposable items, such as drugs, bandages
etc) Need to distinguish between fixed, variable and total cost,
and average, marginal costs and incremental cost
FIXED, VARIABLE AND TOTAL COST
Total cost: all costs incurred while producing a service
Fixed cost - do not vary with quantity in short run e.g. Capital costs
Cost
Quantity
Variable cost : vary with level of service. E.g. consumables
AVERAGE AND MARGINAL COST CURVES
Cost
Quantity
Average cost cost per unit of output
QTC
Marginal costcost of producing an extra unit
QTC
MARGINAL VERSUS INCREMENTAL COST
Cost
Q
MCB, Q
MCA, Q ICA-B, Q
TC of Prog A
TC of Prog B
Quantity
COST ANALYSIS METHODS
1. Cost-effectiveness analysis (CEA):
2. Cost-utility analysis (CUA)
3. Cost-benefit analysis (CBA):
4. Cost-of-illness analysis:
5. Cost-minimization analysis:
6. Cost-consequence analysis:
COST-EFFECTIVENESS ANALYSIS A comparison of costs in monetary units with outcomes in quantitative non-monetary units,
e.g., reduced mortality or morbidity
Target and output indicators pre-decided Most appropriate if the important outcome is uni- dimensional Compare costs of at least two alternatives
Cost- Effectiveness Ratio:
CE Ratio = ($ Cost Int - $ Cost Comp) / ($ Effect Int - $ Effect Comp)
For example: “$45,000 per life- year saved” or $10,000 per lung cancer case averted”
THE COST-EFFECTIVENESS PLANE.
COST-EFFECTIVENESS ANALYSIS CONT…
Advantages
1. Useful to know the cause of failure to attend the objective and the unmet demand.
2. Adequacy of corrective action taken, can be reviewed
Disadvantage
3. Only compare programs with similar outcomes
4. There is not enough information to assign a value to the outcome
5. Methodological inconsistency
6. Ambiguity in assessing overall improvement or decrement in health
7. Cannot address the issue of allocative efficiency
COST-UTILITY ANALYSIS A type of cost-effectiveness analysis It compares costs in monetary units with outcomes in terms of their utility,
usually to the patient, measured, e.g., in QALYs Measures of different effects are consolidated into a common abstract
scale
Example; Quality adjusted life year (QALY), Disablity adjusted life years (DALYs)
Most appropriate if the important outcome is multidimensional
Disadvantage Difficult to assign utility weights
Cost- Utilty Ratio:
CU Ratio = ($ Cost Int - $ Cost Comp) / ($ Utile Int - $ Utile Comp)
For example: “$10,000 per QALY gained”
QALYS GAINED FROM AN INTERVENTION
QUALITY-ADJUSTED LIFE YEARS (QALYS)
Adjust quantity of life years saved to reflect a valuation of the quality of life If healthy QALY = 1 If unhealthy QALY < 1 QALY can be <0
Procedure Identify possible health states - cover all important/relevant dimensions of
QoL Derive utility ‘weights’ for each state Multiply life years (spent in each state) by ‘weight’ for that state.
Quality adjustment utility “weights” should
(1) be preference-based
(2) be interval-scaled
(3) contain both perfect health and death on the same scale
TECHNIQUES TO ‘WEIGHT’ UTILITY Direct methods -
Measure the preferences of individuals directly using general instruments like the visual analogue scale, the time trade-off method, and the standard gamble technique
Widely appliedCostly & time consuming
Indirect methods Simpler to use (though difficult to develop)Based on multi-attribute health status classification systems.
(consist of a set of attributes of health status (e.g., pain and discomfort, visual acuity, ambulation, cognitive function, etc.), and levels of function associated with each attribute from full function to impaired function (e.g. perfect vision — totally blind)
COST-BENEFIT ANALYSIS
It compares costs and benefits, both of which are quantified in common monetary units
Unique feature that can indicate explicitly whether benefits outweigh costs
Advantage: Can address allocative efficiencyCan compare disaperate technologies
Disadvantage: Difficult to assign monetary value to outcomes
COST-BENEFIT ANALYSIS CONT…
Two approaches:
Cost- Benefit Ratio: (Ratio Approach)
CB Ratio = ($ Cost Int - $ Cost Comp) / ($ Benefit Int - $ Benefit Comp)
For example: “Cost Benefit ratio of 1.5”
Cost- Benefit Ratio: (Net Benefit Approach)
CB Ratio = ($ Cost Int - $ Cost Comp) - ($ Effect Int - $ Effect Comp)
For example: “Net cost of $15,00”
MONETARY VALUATION Earlier “Human Capital’ method were used
Willingness To Pay Method Done by Contingency valuation method Assess individual ‘willingness-to-pay’ for (the benefits of) a
good through either:
1. Observed wealth-risk trade-off (revealed preference)Advantage – ‘real’ preferences/valuesDisadvantage – difficult control for confounders
2. Direct survey (stated preference)Advantage – direct valuation of goodDisadvantage – hypothetical/survey problems
Vast majority of CBA use direct survey
COST OF ILLNESS ANALYSIS
A determination of the economic impact of an illness or condition .
Cost of Illness includes: Medical care for prevention, treatment & Social services for rehabilitation Productivity loss
Exampleso What does cancer cost the India?o What does blindness cost the world?
Just because something has the highest cost of illness does not imply that it necessarily should have the most resources directed toward research or cure
Depends on how much it will cost to do something about it
COST - MINIMIZATION ANALYSIS A determination of the least costly among alternative interventions
Calculate the cost of the different methods of achieving the objective Needs at least two ways of achieving the objective
Example
Rabies vaccination: Intramuscular or Intradermal schedule?
Which is least costly?
If so, should we continue Intramuscular schedule or use intradermal schedule?
These studies are difficult because they don’t focus on partial outcomes Need a high degree of certainty that outcome can be obtained or else these
studies are not particularly helpful
SUMMARY OF COST- ANALYSIS METHODS
Valuation of costs
(in Monetary units)
Valuation of
outcomesComments
Cost Effectiveness
$ ÷ Natural unitsIt can only compare technologies whose outcomes are measured in the same units
Cost Benefit $
÷ or –i.e, Ratio Approach or Net Benefit Approach
$
•It enables comparison of disparate technologies•Difficult to assign monetary values to all pertinent outcomes
Cost Utility $ ÷Utilities (e.g.,
QALYs)It enables comparisons of disparate technologies
Cost of Illness $ vs. NoneA determination of the economic impact of an illness or condition .
Cost Minimization
$ vs.Assume same
outcomeA determination of the least costly among alternative interventions
DEALING WITH UNCERTAINTY Sensitivity analysis
◦ Systematically examining the influence of uncertainties in the variables and assumptions employed on the estimated results
Steps:1. Identifying the (uncertain) variables
◦ All variables in the analysis are potential candidates◦ Give reasons for exclusion rather than inclusion
2. Specifying the plausible range over which they should vary◦ Reviewing the literature◦ Consulting expert opinion
3. Recalculating results based on combinations of the best guesses, most and least conservative, usually based on…◦ One-way analysis (each variable separately)◦ Multi-way analysis (number of variables together)◦ Threshold analysis (amount of variance needed to achieve specified result)
ANALYSIS, INTERPRETATION & PRESENTATION
Set up "dummy tables" and graphs at initial phase
Analysis should include interpretation-
i.e. What do the data mean?
Finally, Report the resultStick to objective Keep presentation simple and use tables & graphsWrite an executive summary
STRENGTHS & LIMITATIONS
Strengths o a very useful tool for the manager and policy makero In the absence of certainty, even approximations can help improve
decision-making
Limitations o It's difficult to estimate "true costs" or opportunity costo Allocation of costs; often very difficult
REFERENCES:
Creese A, Parker D. Cost Analysis in Primary Health Care. A training manual for
programme managers. Geneva: World Health Organization; 1994.
Torres T, Baltussen R, Adam T, Hutubessy R, Acharya A, EvansD, Murray C. WHO
Guide to cost effectiveness analysis. Geneva: World Health Organization; 2003.
Hutton G, Rehfuess E. Guidelines for conducting cost–benefit analysis of household
energy and health interventions. Geneva: World Health Organization; 2006.
Module E Cost and Sustainability analysis. Aga Khan Foundation
Cost Analysis Methods. Available from URL: www.nlm.nih.gov/nichsr/
Thank You