The Alberta Health Technologies
Decision Process: Post Policy
Implementation Review
Presenter: Sarah Flynn, Alberta Health
Authors: Dr. Anderson Chuck, Institute of Health Economics; Sarah Flynn, Alberta Health; Dr. Nina Buscemi, Alberta Health; Dr.
Kathryn Ambler, Alberta Health
Disclosure
I have no actual or potential conflict of interest
in relation to this topic or presentation.
2
The Alberta Health Technologies
Decision Process (AHTDP)
• Provincial review process for health technologies
and services that provides evidence and
information to inform decision-making
• A collaboration with Health Technology
Assessment Partners.
• Reviews consider: – effectiveness
– safety
– cost-effectiveness and budget impact
– ethical and legal implications
– patient and provider perspectives
– Potential policy approaches and implications
3
How do we assess impact?
• Health care decisions may be influenced by
clinical impact, cost-effectiveness findings.
• Limited opportunities to actively monitor
implementation of AHTDP-informed policy
decisions.
• Monitoring and evaluation.
4
Post-Policy Implementation Review
(PPIR)
• PPIR is a review of a policy decision to
determine:
• PPIR offers evaluation, accountability
5
Did the policy achieve the
desired results?
Were costs and benefits in line
with expectations?
Were there any unintended or
unforeseen consequences?
What are learnings for future policy development?
Pro
spec
tive
Retro
spective
Retrospective PPIR
6
• Key prerequisites for PPIR:
– Access to information and data
– Readiness to apply the findings
• Some policies are better candidates than others
– Ideal conditions for PPIR
– Policy implemented 2 – 5 years ago
– Clear and logical policy goals
– Adequate studies/evidence available
– Access to original participants
– Commitment of all participants
Testing for Preterm Labour
• Preterm labour/delivery occurs between 20 – 37 wks
• Leading cause of neonatal mortality and morbidity
• Contributes to neurodevelopmental problems,
respiratory/pulmonary dysfunction, hearing and visual
impairment, and other long-term health problems.
• Interventions are available to reduce
morbidity/mortality
• Issue:
– Not all symptomatic women will deliver
– Some women experiencing false labour are being
transferred and admitted to hospital
• Solution: A test which is good at identifying false
labour • The TLiIQ
® System or fetal fibronectin (fFN) testing
8
2006 AHDTP Review
9
• The fFN test was reviewed through the AHTDP to
determine if it should be publicly funded.
• Finding:
– fFN test can aid in ruling out unnecessary
interventions for women in false labour.
– fFN testing would result in cost savings to the
provincial health system through the avoidance of
ambulance transfers and decreased length of
stay.
The Policy Decision (2006)
10
• Introduce fFN testing for preterm labour as a
publicly funded service available to all Alberta
women by 2008
– Regional Health Authorities (RHAs) to implement
their preferred service delivery models and tests.
– Funding from existing budget allocations
– RHAs encouraged to implement quality assurance
mechanisms, guidelines and standards.
• RHAs to chose between fFN and an alternate
test (Actim™ Partus)
PPIR Methodology
11
Is there new evidence on performance differences between fFN and Actim™ Partus?
Literature Review Update
How was implementation conducted?
Barriers, facilitators, and unintended consequences
Key Informant Interviews
Health system impact and costs.
Transfers, length of stay and admissions
Economic Analysis
• Institute of Health Economics conducted a 2015
retrospective PPIR:
Do performance differences exist
between the two tests?
• Both the fFN test and the Actim™ Partus test had
high specificity and negative predictive values.
• Both the fFN test and the Actim™ Partus test had
low sensitivity and positive predictive values.
• FFN was more accurate at predicting preterm
delivery than Actim™ Partus .
• No evidence to suggest that the system adopted
in Alberta (fFN testing) should be changed based
on diagnostic performance
12
Was PTL testing implemented
according to the 2006 policy?
All Regional Health Authorities (RHAs) fully
implemented testing by 2008, using the same test
(fetal fibronectin).
RHAs covered costs through existing budgets*
Staff trained using a variety of materials
Providers generally trust the fFN test and
consider test results as part of their routine for
managing patients with PTL symptoms
Most of the training occurred at policy
implementation; levels of training varied
Access to testing equipment varied
13
Has PTL testing reduced
unnecessary hospital transfers,
admissions, and length of stay?
• Physicians appear to place more significance on
positive test results when deciding to transfer or
admit women (inappropriate use of test).
• Testing did not reduce unnecessary ambulance
transfers or admissions (women in false labour). – Ambulance transfers
• In true labour. 1.91 / 7.45 times more likely to transfer if
+/-
• In false labour: 0.78 / 2.22 times more likely to transfer +/-
– Admissions
• In true labour. 1.68 / 0.44 times more likely to transfer if
+/-
• In false labour: 5.38 / 0.47 times more likely to transfer +/-
14
Has PTL testing had any unintended
consequences?
• More patients being transferred or admitted,
regardless of test results
• fFN testing did not reduce unnecessary
ambulance transfers or admissions for women in
false PTL
• Testing has increased the number of appropriate
ambulance transfers and admissions for preterm
pregnancies in true labour.
15
Has PTL testing resulted in net cost
savings for the system?
• Testing did not reduce unnecessary use of
healthcare services, and associated costs.
• Total health system costs increased due to test
purchases and increased appropriate care due to
testing (more women in true PTL receiving care).
• The increase in health service utilization resulting
from testing has cost the health system an
estimated $3,458,443 for appropriate utilization
and $730,724 for unnecessary utilization
16
What next?
• Shared results with the key provincial stakeholder
group:
– The Maternal, Newborn, Child and Youth (MNCY)
Strategic Clinical Network (SCN)
• Feedback gathered
• The Alberta Advisory Committee on Health
Technologies made a formal recommendation to
the MNCY SCN:
– Do not maintain the status quo
– Consider the PPIR and Policy Analysis results
– Develop a plan of action to address inappropriate use of
preterm labour testing.
• March 20 decision to discontinue testing
17
Reflections
18
First PPIR of an AHTDP Decision:
• Challenges of a retrospective approach
• Working from an AHTDP review
• Capitalizing on HTA competencies
• Lessons for future reviews
Lessons
19
• Strengthening AHTDP reviews:
– Draw on HTA findings to develop comprehensive
policy options and recommendations
– Leverage HTA findings to support strategic,
measured implementation
• Strengthening PPIR:
– Identify a key stakeholder or “client”
– Prospectively evaluate or review policy decisions,
where possible
Top Related