Describing the quality of New Zealand’s health and disability services Developing our Health Quality and Safety Indicators

Engagement and feedback document

Published in July 2012 by the Health Quality & Safety Commission, PO Box 25496, Wellington 6146

This document is available on the Health Quality & Safety Commission website: www.hqsc.govt.nz

For information on this report please contact info@hqsc.govt.nz

Contents

Executive Summary iii

1. Introduction 5 1.1 Purpose 5 1.2 How can I provide feedback? 5

2. Why develop quality and safety indicators for New Zealand?7

2.1 What do we mean by indicators? 7 2.2 Measurement and indicators as part of the Commission’s toolkit 7

3. Definition of the indicator framework 9 3.1 What do we seek to achieve? 9

3.1.1 Our goal 93.1.2 Our objectives 9

3.2 What will the indicators address? 93.2.1 What level of measurement? 93.2.2 What services are within the scope? 103.2.3 Which dimensions of quality are of interest? 10

3.3 Who are the audiences? 10

4. Identifying the initial indicator set 12 4.1 Overview of approach 12

4.1.1 Project structure 124.1.2 Process 12

4.2 Summary list of proposed indicators and measures 13 4.3 Achieving a balanced set of indicators 15

5. Quality and safety indicators – initial results 17 5.1 Cancellation of elective surgery by the hospital 17 5.2 Amenable mortality 19 5.3 Ambulatory sensitive hospitalisation 22 5.4 Occupied bed-days for people aged 75 and over admitted two or

more times per year 24 5.5 Day case surgery turns into unplanned overnight stay 26 5.6 Hospital unplanned and unexpected readmission 28 5.7 Eligible population up-to-date with cervical screening 30 5.8 Age appropriate vaccinations for two year olds 32 5.9 Health care cost per capita (US$ Purchasing Power Parity per

capita) 33 5.10 Health care expenditure as a proportion of GDP33

6. Further indicator development 35

Appendices

Appendix 1 – Expert Advisory Group...............................................................36

Executive SummaryThis report presents the first set of health quality and safety indicators to be published by the New Zealand Health Quality & Safety Commission (the Commission) and seeks feedback from interested parties on the overall set.Publishing a set of health quality and safety indicators is a core part of the Commission’s work and is required under the New Zealand Public Health & Disability Amendment Act 2010, where the Commission is charged with:

providing advice to the Minister of Health on how quality and safety in health and disability support services may be improved

leading and coordinating improvements in quality and safety in health care

identifying key health and safety indicators (such as events resulting in injury or death) to inform and monitor improvements in quality and safety

reporting publicly on quality and safety, including performance against national indicators

sharing knowledge about and advocating for quality and safety.In developing these indicators the Commission has built upon existing quality and safety initiatives throughout the sector, and involved measurement experts and stakeholders in the process of selection and definition. The Commission has also sought to select indicators across a wide range of dimensions of health care delivery and consumer experience of health care. This report presents the work towards developing 17 proposed indicators, findings where available, and seeks feedback and suggestions on how this indicator set could be further developed and strengthened. Structure of Report Section 1 provides details on how interested parties can provide feedback, listing the questions on which the Commission is most interested in receiving feedback. Sections 2 and 3 focus on explaining the international context in which health quality and safety indicators have been developed and how they fit with other work currently being undertaken by the Commission. These sections also provide the particular rationale and aims of indicator development in the New Zealand context, and describe possible audiences and their likely interest in the measures. Section 4 sets out the process taken by the Commission to develop and publish the first set of indicators. Indicators have been categorised according to:

how ready they are for publication. This depends upon the robustness of the indicator and whether it is already in use or under development and on availability of data. Indicators are categorised as either fast-track, under development or placeholder

whether they are a system level indicator or a contributory measure. System level indicators are headline indicators that provide a balanced picture of the status of the quality and safety of health and disability support services in New Zealand. Contributory measures focus on reporting on outcomes of health care delivery for defined patient populations, specific quality activities or services delivered

the overall focus (whether on safety, patient experience, effectiveness, access, efficiency or on equity).

Section 5 presents our initial findings, where these are available.

If you wish to provide feedback on the indicators, you can

Complete an online feedback form on the Health Quality & Safety Commission’s website - www.hqsc.govt.nz

Participate in regional meetings. The “roadshow” will be held in October – details will be available on the Commission’s website.

From early August Definition Standards will be available from the Health Quality Measures NZ website: http://www.patientsfirst.org.nz/hqmnz

The Definition Standards provide detailed information on the definition and calculation of each indicator

All feedback provided to the Healthcare Quality Measures Library will be analysed as part of the feedback process.

In order to be included in the initial analysis, please provide your feedback by Friday 14 September 2012.

1. Introduction

1.1 PurposeThe New Zealand Health Quality & Safety Commission (the Commission) was established in 2010 to lead and coordinate work on monitoring and improving the quality and safety of health and disability support services to ensure all New Zealanders receive the best health and disability care within our available resources.As part of its work on measurement and evaluation, the Commission develops and regularly publish a set of indicators to drive improvement of the quality and safety of health and disability support services provided within New Zealand. It seeks to do this in a way that complements and builds on existing initiatives and learns from and involves stakeholders and key experts in the field of quality measurement. This document is the Commission’s first publication of the proposed set of health quality and safety indicators (‘HQSIs’ or the ‘indicators’).

In this report we:

explain why we are developing the indicators and what we seek to achieve

set out our thinking and the development process behind the creation of the draft set of indicators

provide details for 17 proposed indicators, including 12 ‘fast-track’ indicators (where there is an existing defined and tested indicator)

publish preliminary results for those indicators where data is readily available

identify the other areas where we believe indicators are required, which will be developed in the future

publish available indicators

engage the sector and stimulate debate to encourage feedback on our proposed approach and set of indicators.

1.2 How can I provide feedback?We are interested in your views on our initial indicator set, how these indicators could be used, and our overall approach to developing the full indicator set in the longer term. Your feedback will help shape this process and ensure indicator information will be applied to improve the quality and safety of New Zealand’s health services.

We are specifically interested in your answers to the following questions.

1. Do you agree with the stated purpose of the indicators?

2. Is the range of topics covered in the scope of the indicator set wide enough?

3. What, if any, other services or quality dimensions would you include?

4. What, if any, barriers do you envisage in reporting quality and safety indicator information?

5. How could the reporting of the quality and safety indicators be facilitated or supported?

6. What use is the information generated by the initial set of indicators to you? What impact will it have on your activities?

7. Which, if any, of the indicators in the initial set are most useful? Why?

8. Which, if any, of the indicators in the initial set do you see as problematic? Can you explain why?

9. Are there any other indicators you think should be added to the initial set? If so, which ones and why?

10. Are there any indicators or topics you think should be included in future sets of indicators? If so, what are they and why?

11. Are you interested in helping the Commission develop its quality and safety indicator set? If you are, please provide details of the perspective or expertise you could contribute and your preferred contact details.

2. Why develop quality and safety indicators for New Zealand?

2.1 What do we mean by indicators?Over recent decades, there has been a growing focus in the health care systems of developed countries on measuring health care quality through national and local performance reporting. A recent King’s Fund report1 contained the following quote:

‘We can only be sure to improve what we can actually measure.’

There are a range of definitions and understandings of what is meant by various terms used in the field of quality improvement, measurement and evaluation. We have adopted the definition from the NHS Institute for Innovation and Improvement (2008)2 to describe our understanding of the term, as outlined below:

‘An indicator is a summary measure that aims to describe in a few numbers as much detail as possible about a system, to help understand, compare, predict, improve, and innovate.’

2.2 Measurement and indicators as part of the Commission’s toolkitInternational literature provides 20 years of evidence that measuring the quality of health care and communicating the results in a variety of ways and settings is a powerful way to stimulate improvement in health care. However, the literature also suggests that maximising this effect requires a clear understanding of the purpose of the measurement, which in turn should influence the precise nature of the measurement.

In the New Zealand context, this prompts a number of questions

Are we interested in the average performance of the overall system, or in the variations occurring between different areas or institutions?

Are we seeking to use measures to make an absolute judgement or to ask valuable questions?

Are we seeking primarily to provide accountability or to stimulate improvement? Does this information relate to populations, institutions or to individuals?

Measurement and evaluation is a critical part of the Commission’s work, underpinning and forming the basis of some specific work programmes, as outlined below.

Atlas of Healthcare Variation: Concentrating on individual conditions and clinical groups, the Atlas highlights variation in order to stimulate discussion about difference in practice and the improvement actions required to eliminate these where unwarranted.

1 Raleigh SV, Foot C. 2010. Getting the measure of quality – Opportunities and challenges. The King’s Fund: 2010.2 NHS Institute for Innovation and Improvement. 2008. The good indicators guide: understanding how to use and choose indicators. Coventry: NHS IIIC.

Quality and safety markers: These markers are a mix of process and outcome measures focused on reducing harm from in-patient falls, hospital-acquired infections, surgery and medication. They do this through the setting of expected levels of improvement, public reporting of progress against these thresholds and links to accountability mechanisms.

Measurement and evaluation of improvement programmes: The Commission supports a range of specific quality and safety improvement programmes (such as the Medication Safety Programme which aims to reduce harm from medication errors and increase the efficiency and safety of medication management systems in the sector). The Commission shall approach the design and establishment of such programmes with a clear view of the desired outcomes and the approach to measuring change.

This report: These are a small set of summary indicators which provide the public and the health and disability sector with a clear picture of the quality and safety of health and disability services in New Zealand, including changes over time.

3. Definition of the indicator framework

3.1 What do we seek to achieve?

3.1.1 Our goalThe over-arching goal of reporting against a set of quality and safety indicators is to provide robust information to support achievement and measure progress against delivery of the outcomes articulated in the New Zealand Triple Aim framework, namely:• improved quality, safety and experience of care• improved health and equity for all populations• best value from public health system resources.

3.1.2 Our objectivesDeveloping a set of credible, robust and reliable quality and safety indicators will help us achieve the following objectives:• provide the public and the health and disability sector with a clear picture of the

quality and safety of health and disability services in New Zealand, including changes over time

• inform the quality improvement activities of service providers by providing information to support learning and peer review in clinical settings

• identify key quality and safety issues and prioritise improvements to the quality and safety of health and disability support services.

3.2 What will the indicators address?In this section we describe the scope of services and quality domains that the indicators will cover. (Figure 1 illustrates these dimensions and shows how they all fit together.)

Over time there are likely to be changes to the set, as definitions for existing indicators are refined, as new indicators are added (reflecting priorities identified by the sector or determined through the Commission’s work programme) and as others are ’retired‘ as they become less relevant.

The first set of indicators published will be unable to provide coverage across the entire scope of health care. We are proposing a pragmatic approach, starting with a relatively small set of indicators to get the process rolling and to test the framework. It is likely the number of indicators will increase as coverage becomes more comprehensive.

3.2.1 What level of measurement?In designing the framework for the indicator set, our key sector stakeholders from the expert advisory group advocated a two-tier approach.• System-level indicators which provide a balanced view of the overall quality and

safety status of New Zealand’s health and disability support services. • Contributory measures – which relate to the delivery of specific health care

services, the implementation of quality initiatives or outcomes for specific population groups.

3.2.2 What services are within the scope?The scope of the indicator set seeks to cover services throughout the patient’s journey that are provided across the entire health and disability sector, including:

public, private and non-government organisation (NGO) providers primary care, hospital, aged care, mental health and disability support sectors.

It is not essential that all indicators and contributory measures can be applied across every type of service provider. However, as we continue to develop the quality and safety indicators we will seek to ensure a balanced spread of indicators that reflect all service areas within the defined scope. (We recognise our first proposed set of indicators has a bias towards hospital care, although some of the measures can act as proxies for the effective coordination of care across the whole system.)

3.2.3 Which dimensions of quality are of interest?The three outcome areas of the New Zealand Triple Aim provide the foundations for the indicator framework. These reflect a broad interpretation of ‘quality’ across the system, population and patient. As such, the set of indicators selected should represent a balanced spread relating to all contributing dimensions of quality including safety, patient experience, effectiveness, access, efficiency and equity.

3.3 Who are the audiences?A range of potential audiences will have different interests and requirements from the information provided, as outlined in Table 1 below.

Table 1: Audience interests and requirements

Some key audiences

Inform quality improvement

activities to support learning

in clinical settings

Provide a picture of

quality and safety of

health services

Identify priorities for

improving quality and

safety Specific interests and requirements

Members of the public

Clear, concise, accessible information that tells a story.

Clinicians & other service providers

Primary interest likely to be in measures directly relevant to clinical/service activities. Need to win ‘hearts and minds’ by ensuring information is robust and relevant.

Private service providers, eg, nursing homes & NGOs

District health boards

Interests from range of perspectives/different parts of organisation.

Ministry of Health and Ministers

Alignment/improvement against priorities.

Figure 1: – The Health Quality and Safety Indicator framework

4. Identifying the initial indicator set

4.1 Overview of approach

4.1.1 Project structureThe Health Quality and Safety Indicator project (the HQSI project) was established to oversee the development and implementation of the initial set of indicators.

A Project Steering Group from the Commission has been working closely with an External Advisory Group (EAG) made up from key stakeholders and sector experts. (Membership of the EAG is listed at Appendix 1.)

4.1.2 Process

Guiding principlesWhen developing the initial indicator set, the Commission wanted to build on existing work in the field of quality and safety measurement both in New Zealand and internationally and to minimise the reporting burden on the sector. As a result wherever feasible and relevant we sought to:

• use indicators that have already been developed, defined and tested• use existing data collections• align indicators to other parts of the Commission work programme• select some international measures to enable comparability with other countries.

These principles underpinned the following staged process.

Stage One: Review of Institute for Healthcare Improvement (IHI) Whole System MeasuresThe IHI Whole System Measures3 were developed as a balanced set of metrics to measure the overall quality of a health system. This approach is strongly aligned to the purpose envisaged for the New Zealand HQSI indicators.

As we developed our initial set of indicators, we reviewed the IHI measures as a ‘start-point’ and used them as a ‘straw man’ to test coverage of our key areas of interest and relevance in a New Zealand context. This resulted in several IHI measures being adopted and adapted for the first indicator set.

Other measures were identified as useful for the development of future indicators, but require further work on identifying and assessing relevant sources of data.

Stage Two: Identifying existing quality and safety indicatorsThe project team reviewed New Zealand and overseas indicator frameworks to identify and assess how existing indicators met the HQSI purpose.

Indicators that fell within this scope included those from the:

• New Zealand District Health Board Hospital Quality and Productivity programme• New Zealand Primary Health Organisation (PHO) performance programme3 Martin LA, Nelson EC, Lloyd RC, Nolan TW. 2007. Whole System Measures. IHI Innovation Series white paper. Cambridge, Massachusetts: Institute for Healthcare Improvement. (Available on www.IHI.org).

• Australian Quality and Safety Commission • Commission’s quality improvement work programme.

It was agreed that creating a composite set of quality and safety indicators from these sources (which generally have a wider focus than quality and safety) would be valuable and make possible the telling of a different story.

Stage Three: Narrowing down to the initial set of indicators Stages One and Two identified a long list of potential indicators. Through workshops and discussion our EAG and Project Steering Group narrowed the list down to a set of 23 measures.

The EAG then scored each potential indicator against seven strategic criteria which enabled us to prioritise each indicator and identify the consolidated initial working list of 17 indicators.

4.2 Summary list of proposed indicators and measuresTable 2 presents a summary of our proposed working list of system-level indicators and contributory measures. These are colour-coded, according to the key below with the different colours indicating the level of preparedness for implementation.

There are a total of 17 indicators, of which 12 either exist already or are easily sourced from routine data and five that require further work to define, and in some cases collect, the data.

In addition, there are a further seven areas where we know we want an indicator of some description, but currently there is no plausible data available or we are uncertain about what would make a suitable measure.

KeyFast-track – An existing defined and tested indicator; there is likely to be good availability of data from existing collections. In some cases these will be collected as indicators, in others they will be derivable from existing data sets.

Under development – Work underway to define indicator; further work required to understand availability of data.

Placeholder – Important area but significant further work required during next phase to develop indicator and derive data.

Table 2: Working list of system-level indicators and contributory measures

SYSTEM LEVEL INDICATORS CONTRIBUTORY MEASURESSAFETY MEASURESAdverse events Placeholder: system-level measure of

adverse events (potentially a composite of contributory measures)

1. Under-development: Number of falls resulting in harm in hospitals

2. Under-development: Health care-acquired infection

3. Under-development: Measure of surgical harm

4. Under-development: Measure of safe medication management

Placeholder: Pressure ulcers acquired in hospitals

PATIENT EXPERIENCE MEASURES5. Fast-track: Cancellation of elective

surgery by the hospitalPlaceholder: Measure of patient experience of care

EFFECTIVENESS MEASURESImpact on health outcomes

6. Fast-track: Amenable mortalityPlaceholder: functional health outcomes scores (eg, patient reported outcomes such as EQ5D, SF26)

Effectiveness of care 7. Fast track: Ambulatory sensitive (avoidable) hospitalisation

8. Fast-track: Occupied bed days for people aged 75+ admitted two or more times per year

9. Fast-track: Day case surgery turns into unplanned overnight stay

10. Fast-track: Hospital unplanned and unexpected readmissions

11. Fast-track: Mental health readmission and length of stay

Chronic conditions 12. Under development: Measure of CVD management

Placeholder: Measure of diabetes management

ACCESS MEASURESAccess to preventative care

13. Fast track: Eligible population up-to-date with cervical screening

14. Fast track: Age appropriate vaccinations for two year olds (and potentially also at eight months)

Access to primary health care

Placeholder: system-level measure of access to primary health care (eg, ability to enrol with PHO and/or babies enrolled with a PHO in first three months of life)

Efficiency measuresResource utilization 15. Fast-track: Health care cost per

capita (US$ Purchasing Power Parity per capita

16. Fast-track: Health care expenditure as a proportion of GDP

17. Fast-track: Hospital days during last six months of life

EQUITY MEASURESUnder development: Stratification of all measures across population groups

4.3 Achieving a balanced set of indicators Figure 2 maps the proposed system-level indicators and contributory measures by the triple aim outcomes and quality domains.

It demonstrates there are comparatively few quality domains where we have readily-available system indicators, but that we are in a stronger position for contributory measures. We are able to measure something valuable in all the domains of quality, but single summary indicators are more complex. For the safety domain we may wish to consider a composite of the contributory measures. For the patient experience domain, we will be reliant on the availability of a robust consumer experience survey. For effectiveness and efficiency, our position is more robust. However, access is more complicated because of the issue of the coverage of different sectors.

Figure 2– Indicators and measures by quality domain

5. Quality and safety indicators – initial results

In this section we present ten of the fast-track indicators, with an outline of the indicator definition, the rationale for selection and a brief overview of preliminary results obtained from analysis of the data.

Further detail on the definition and calculation of each of the indicators is provided in the definition standards found on the HQMNZ website: http://www.patientsfirst.org.nz/hqmnz”

5.1 Cancellation of elective surgery by the hospital

What does this indicator measure?This indicator measures the percentage of elective surgeries (excluding maternity surgeries) that were cancelled by the hospital after the patient had been admitted.

Why are we interested?Cancellation of surgery once the patient is admitted will provide insight into how close to capacity the system is running and is a measure of patient experience that has been found to be important to patients in other health systems. Use of this indicator would enable national comparison on continuity of care/ system performance and issues of capacity and patient experience.

What do the figures show?The figures show that about one percent of operations are cancelled after admission and that this proportion has been relatively consistent across the country over the past four years. While this may seem like a small percentage of total operations, it actually amounts to some 5,000 cancellations in New Zealand per year. This represents both a significant level of resource and also considerable disruption to patients. We note also that there is significant variation around the country, with a nine-fold difference between the highest four-year average level of cancellations (2.7%) and the lowest (0.3%).

Figure 3 : Rates of operations cancelled after admission by year 2008–2011

2008 2009 2010 2011 2008-20110%

1%

2%

3%

New ZealandLowest DHBHighest DHB

Rate

s of c

ance

lled

oper

ation

s

Source: HQSC analysis of National Minimum Dataset data

What are the important considerations or potential caveats?We recognise hospital capacity requires close management and that this analysis does not take into account the reason for cancellation. There will be a seasonal impact, with medical acute conditions likely to dominate during winter meaning that fewer beds are available for elective surgical cases. The onset of epidemics can be relatively unpredictable.

That said, we believe this remains a valid measure of patient experience and indeed of the efficiency of related processes. It recognises the need for hospitals to have in place systems to manage capacity issues in relation to the interface between acute and planned care, in a timely and efficient way.

5.2 Amenable mortality

What does this indicator measure?This indicator is effectively a measure of deaths that are potentially avoidable through the timely provision of appropriate health care.

The Ministry of Health currently defines amenable mortality as deaths from those conditions for which variation in mortality rates (over time or across populations) reflects variation in the coverage and quality of health care (itself defined as preventive or therapeutic services delivered to individuals or families).4

The definition covers all deaths under 75 years of age (by age, sex, ethnicity, deprivation) from a specified list of 35 conditions (or groups of related conditions) classified into six categories: infections; injuries; maternal and infant conditions; cancers; cardiovascular diseases and diabetes; and other chronic diseases.

Why are we interested?Amenable mortality is a well-tested indicator that has been consulted upon widely and is accepted in academic circles as a whole-of-system health outcome indicator.

In the area of effectiveness, amenable mortality helps to answer questions around the potential for gain in health outcome and the cost effectiveness of the health system: What health gain has resulted from the increase in health expenditure?5

It is suitable as a whole-of-system health outcome indicator for use in health system performance assessment.

What do the figures show?Figures 4 and 5 both compare New Zealand’s age-standardised amenable mortality rates for those aged up to 75 years to overseas jurisdictions.

Figure 4 compares our rate with that of Australia for the years 1997to 2006, while Figure 5 presents World Health Organisation mortality data for the 16 High Income Nations, from research conducted by the Commonwealth Fund. Please note the calculation for amenable mortality is slightly different in the two analyses. Nevertheless it can be seen that New Zealand’s amenable rates are the fourth highest among High Income Nations, and while trending down in the same way, are higher than Australian rates.6

4 http://www.health.govt.nz/publication/saving-lives-amenable-mortality-new-zealand-1996-2006

5 Over the past decade, health expenditure in New Zealand funded via Vote: Health has grown at an annual rate of 4.9 percent in real terms (3.7% per capita). See: http://www.health.govt.nz/publication/saving-lives-amenable-mortality-new-zealand-1996-2006 .

6 Nolte E, McKee M. 2011 Variations in Amenable Mortality – Trends in 16 High-Income Nations, The Commonwealth Fund, September 2011, http://www.commonwealthfund.org/Publications/In-the-Literature/2011/Sep/Variations-in-Amenable-Mortality.aspx

Figure 4: New Zealand and Australian age-standardised amenable mortality rates by year 1997–2006

1997 1998 1999 2000 2001 2002 2003 2004 2005 20060

20

40

60

80

100

120

140

160Australia New Zealand

Deat

hs p

er 1

00,0

00 p

opul

ation

Source: http://www.health.govt.nz/publication/saving-lives-amenable-mortality-new-zealand-1996-2006

Figure 5: Countries age-standardised amenable mortality rates for under 75 years, for the years 1997–98 and 2006–07

Fran

ce

Aust

ralia

Italy

Japa

n

Swed

en

Norw

ay

Neth

erla

nds

Aust

ria

Finl

and

Germ

any

Gree

ce

Irela

nd

New

Zea

land

Denm

ark

Unite

d Ki

ngdo

m

Unite

d St

ates

0

20

40

60

80

100

120

140

1601997/8 2006/7

Deat

hs p

er 1

00,0

00 p

opul

ation

Source: Nolte E, McKee M. 2011 Variations in Amenable Mortality – Trends in 16 High-Income Nations, The Commonwealth Fund, September 2011, http://www.commonwealthfund.org/Publications/In-the-Literature/2011/Sep/Variations-in-Amenable-Mortality.aspx

Figure 6 illustrates how the rate of amenable deaths varies by gender and age. As can be seen, the amenable death rate is higher for males than females, particularly with increasing age, and is much higher for those in the 65 to 75 age group.

Figure 6: Amenable deaths per 100,000 deaths in 2006 by gender and age group

0–14 15–24 25–44 45–64 65–740

100200300400500600700800900

1000

MaleFemale

Age groups (up to 75 years)

Amen

able

mor

talit

y ra

te /

100

,000

de

aths

Source: http://www.health.govt.nz/publication/saving-lives-amenable-mortality-new-zealand-1996-2006

What are the important considerations or potential caveats?Due to relatively small numbers we have deliberately avoided presenting subnational variation for this indicator. In smaller DHBs particularly, the rates have wide uncertainty intervals which makes interpretation of variation more complex.

However, for the purposes of the quality and safety indicator set we intend to report this indicator at a national level, providing opportunity for international comparisons.

Further, monitoring trends will be more informative than a single point-in-time analysis, although the timeliness of cause of death data is currently problematic. Unlike avoidable hospitalisations (for example), which can be monitored quarterly with a rolling six-month delay, avoidable mortality can never be a real-time indicator.

5.3 Ambulatory sensitive hospitalisation

What does this indicator measure?Ambulatory sensitive hospitalisations (ASH) are those admissions (mostly acute) that are considered by expert opinion to be potentially preventable through interventions in out-of-hospital settings.

It is defined as hospitalisations of people less than 75 years old resulting from diseases sensitive to prophylactic or therapeutic interventions that are deliverable in a primary health care setting. Interventions which may reduce ASH include programmes for primary prevention, community interventions, screening and diagnostic services, and specific treatments for individuals in different age groups.

Ambulatory sensitive hospitalisations were included as a National Health Target from July 2007.

Why are we interested?The rate of ambulatory sensitive hospital admissions is often used as a measure of the effectiveness of the interface between primary and secondary health care. The assumption is that better management of chronic conditions such as diabetes and cardiovascular disease within local communities has the potential to reduce the number of avoidable hospital admissions (and to moderate demand on hospital resources).7 If there is good access to effective primary health care for all population groups, then it is reasonable to expect that there will be lower levels of ambulatory sensitive hospital admissions.

This indicator can also highlight variation between different population groups that will assist with DHB planning to reduce disparities.

7 http://www.treasury.govt.nz/publications/informationreleases/health/primaryhealthcare/ddd-primhealthcare-may08.pdf

What do the figures show?

Figure 7: Ambulatory Sensitive Hospitalisation rates per 100,000 admissions by year

2006/07 2007/08 2008/09 2009/10 2010/11 2011 Min Max 1 July - 30 June 1 Jan - 30 December

0

500

1,000

1,500

2,000

2,500

3,000

Ambu

lato

ry se

nsiti

ve h

ospi

talis

ation

rate

s per

10

0,00

0 po

pula

tion

Source: MoH analysis of NMDS

What is clear is that there has been remarkably little change in the ASH rates over the last five years. If the hypothesis that these hospitalisations are amenable to change based upon out of hospital interventions is correct, this lack of change is of note. However the variation between DHBs, shown in the Min and Max bars is striking with the highest ASH rates being approximately twice the lowest.

What are the important considerations or potential caveats?There has been much debate about the usefulness of Ambulatory Sensitive Hospitalisations as a measure of access to services and effectiveness of system performance.

There is some evidence of its validity as a proxy indicator of access8 9, but this remains a contested and controversial indicator.

From a practical perspective, measuring change over time requires consistency in definition of both the basket of conditions and exclusions from the underlying data. Periodically there have been changes to the definition of this indicator. As such, it is important to ensure comparability of data presented according to the same definition.

8 Ansari 2007). The concept and usefulnesss of Ambulatory Care Sensitive Conditions as Indicators of Quality and Access to Primary Health Care Australian Journal of Primary Health 13(3) 91-110

9 HSAC (2008) The effectiveness of interventions for reducting ambulatory sensitive hospitalsisations: a systematic review

5.4 Occupied bed-days for people aged 75 and over admitted two or more times per year

What does this indicator measure?This is the rate of occupied bed days for people aged 75 and over who had more than two emergency admissions in a financial year per 1,000 population aged 75 and over. The definition excludes both elective and mental health admissions.

Why are we interested?This measure is shown in both England and Scotland10 to be a useful proxy for the effectiveness of integration of primary, acute and long stay care, showing both effectiveness of avoiding unnecessary admissions, and ability to ‘step down’ to less intensive forms of care.

This measure offers the potential to identify opportunities to reduce the number of admissions and/or length of stay by providing effective alternative care at home or in residential care settings. Evidence from England suggests achievable elimination of variation would have released c. $4bn (equivalent to 2.5% of the NHS budget) back into the NHS.

What do the figures show?As can be seen in Figure 8, page x the number of occupied bed days for people aged 75 and over who are admitted more than once are year in New Zealand is relatively low when compared with the United Kingdom (the other country with a consistent time series for this particular measure). In New Zealand the rate is around 1250 bed days per 1000 people aged 75 and over, compared to around 2000 in England. In addition the relative variation in the rates is considerably smaller, at around a two to three fold variation, than the six fold variation seen in England. Nevertheless variation of this scale raises the question of whether it could be reduced by widespread adoption of the integration practices seen in the areas with the lowest rates.

10 Currie C. 2010. Health and Social Care of Older People: Could Policy Generalise Good Practice?, Journal of Integrated Care, Vol. 18 Iss: 6, pp.19 – 26.

Figure 8: Occupied bed-days for people aged 75 and over, admitted two or more times per 1000 population

2008/09 2009/10 2010/110

500

1000

1500

2000

2500AverageMinimumMaximum

Occ

upie

d be

d da

ys a

ssoc

iata

ed w

ith p

eopl

e ag

ed 7

5+

adm

itted

twice

or m

ore

as a

n em

erge

ncy

per 1

,000

pop

-ul

ation

Source HQSC analysis of NMDS

What are the important considerations or potential caveats?Some variation seen in the results of this measure may be related to demographic factors.

The numerator definition represents a slight understatement of the actual position because patients may be admitted more than twice within a rolling year, however if those admissions are either side of the calendar year end, they will not be included. It is important to note that this measure is affected by organisation of all aspects of health and aged care; a high number of occupied bed days cannot be attributed to the actions of the hospitals alone but also of access to, and quality of, primary, in-home and residential care, and the degree of successful integration between all four.

5.5 Day case surgery turns into unplanned overnight stay

What does this indicator measure?This indicator measures the proportion of planned day case admissions that turn into overnight stays.

Why are we interested?Day case overstays are an outcome and efficiency measure of secondary care at a surgical department level.

An unplanned overnight stay from a day case is always a diversion from the ‘planned approach to care’, even if we do not know why the extended stay occurred (and there may be a very good clinical reason for the change).

While the measure reflects a change in plan (a cause of inconvenience to patients, disruption to planned hospital flow, etc) it is an area where both cause and proper response is uncertain. It may reflect an adverse incident in a procedure, unrealistic assessment about what may be treated as a day case, or some other local factor. This makes it a rare sort of indicator that both raises questions and provides an assessment of performance, with both quality and efficiency dimensions.

What do the figures show?

Figure 9: Day case overnight stay rate 2008/09- 2010/11

2008/09 2009/10 2010/110.0%

5.0%

10.0%

15.0%

20.0%

25.0%

30.0% New Zealand average

Minimum rate >1000 DCs

Maximum rate >1000DCs

Day

case

ove

rsta

y ra

te

Source HQSC analysis of NMDS

Nationally, the proportion of day cases that turn into unplanned overnight stays has remained consistent in recent years, while the most extreme rates have fallen dramatically over the last three years. This particular measure has been used in England for some time and the most recent reported data from 2009 show a similar national average rate. Nevertheless, on the face of it, this figure equates to around 10,000 people a year who expected to be in and out of hospital in a day who had to make an overnight stay.

What are the important considerations or potential caveats?As noted above, this measure does not identify the reasons for an overstay and there may be very legitimate clinical reasons for keeping patients overnight. Hence, the results need to be interpreted with caution.

We would anticipate variation in results between DHBs in relation to demographic or geographic factors (for example, in rural settings a potentially longer distance to hospital may affect ability to travel within the same day). As such, the results should be presented at a national level.

5.6 Hospital unplanned and unexpected readmission

What does this indicator measure?

This measures the percentage (standardised) of discharges readmitted within a set period (attributed to the DHB of the initial discharge). The denominator excludes zero or one day stay initial admissions and ACC funded initial events.

Why are we interested?Acute readmissions are viewed negatively as being linked to complications from inadequate care. The responsibility is placed with the initial facility/DHB.

This measure can be a proxy of both the care received in the hospital and the coordination of care back to the community setting and within the community setting. While some readmissions are part of the planned care and are desirable, others may be an indication of a quality issue related to inappropriate shortened length of stay and premature discharge, inadequate care, or lack of patient adherence to the care regimen following discharge from the hospital. It is important to include boundary-crossing measures such as this, highlighting the interaction between the different parts of the system.

What do the figures show?The figures presented here illustrate a consistent trend of increasing volumes of readmission between 2007 and 2011. This represents an increase in readmission rate from 8.0 percent in 2007 to 9.2 percent in 2011. The most recent rates internationally suggest that New Zealand’s readmission rate is fairly typical although precise definitions of the indicator vary between countries. England has shown a substantial increase in readmission rates in the last ten years, and these stand at around 11 percent 11. Similarly recent data from Canada suggest a readmission rate of around 8 percent12.

11 http://www.ic.nhs.uk/statistics-and-data-collections/hospital-care/accident-and-emergency-hospital-episode-statistics-hes/hospital-episode-statistics-emergency-readmissions-to-hospital-within-28-days-of-discharge

12 http://www.cihi.ca/CIHI-ext-portal/pptx/internet/readmission_ac_return_to_ed_en

Figure 10: Percentage of hospital admissions followed by an unexpected or unexplained readmission within 30 days of discharge

2007 2008 2009 2010 2011-2%

0%

2%

4%

6%

8%

10%

12%

14% All NZ

Lowest DHB

Highest DHB

Perc

enta

ge o

f hos

pita

l adm

issio

ns

follo

wed

by

unex

pect

ed re

adm

issio

n w

ithin

30

days

of d

ischa

rge

Source: HQSC analysis of NMDS

What are the important considerations or potential caveats?This is a fairly crude indicator that does not take into account the nature of unplanned readmission, and indeed whether there is appropriate care available in the community that may have prevented the need for admission.

It is likely to be influenced by demographic factors, such as the proportion of older people within a district population, and by existing levels of co-morbidity.

5.7 Eligible population up-to-date with cervical screening

What does this indicator measure?This measures the percentage of eligible women (aged 20 to 69 years) who have received a cervical smear in the past three years. This measure has been part of the PHO Performance Programme's National Indicator Set since 1 January 2011.

Why are we interested?Early detection and treatment of cervical cancer and other abnormalities lowers the rate of premature death for women. The available international evidence suggests that women between the ages of 20 and 69 years should be screened for cervical cancer once every three years.

Coverage of cervical cancer screening refers to women who have enrolled in the programme and have had a smear in the past three years. Coverage is related to protection: a woman who is covered has her risk of cervical cancer reduced by about 90 percent.

This measure can be viewed as an indication of levels of access to primary health care services. It also considers a preventative service, which is not reflected in any other proposed indicator.

What do the figures show?Cervical screening rates in New Zealand are high in comparison with other countries, with only Norway and the UK recording higher screening rates13. Whilst this is clearly encouraging, there is substantial variation between PHOs, and there is generally a lower rate of screening amongst “high-need” women.

Figure 11: Percentage of all and high needs women aged 20-65 in 2010 who were eligible and up-to-date with cervical screening

All women aged 20-69 High need women aged 20-690%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

New Zealand Lowest PHOHighest PHO

Perc

enta

ge o

f elig

ible

wom

en u

p to

da

te w

ith ce

rvica

l scr

eeni

ng

Source: Ministry of Health

What are the important considerations or potential caveats?Many women who have had a hysterectomy do not need a cervical smear. The PHO Performance Programme applies an adjustment calculation to allow for women with

13 Machii and Saito Time Trends in Cervical Cancer Screening Rates in the OECD Countries Jpn. J. Clin. Oncol. (2011) 41 (5): 731-732

hysterectomies, based on the national rate. However since the rate of hysterectomies within each PHO may vary, this adjustment may not always be correct at the PHO level.

Some patients choose to ‘opt off’ the national screening programme’s register (which means that although they have had a cervical screen, they will not be ‘counted’ by the Programme).

5.8 Age appropriate vaccinations for two-year-olds

What does this indicator measure?The number of children within the two-year-old age cohort (between 24-35 months of age) who have received complete set of age appropriate vaccinations.

This measure has been part of the PHO Performance Programme's National Indicator Set since 1 January 2006.

Why are we interested?Children who receive the complete set of age appropriate vaccinations (in this case for the two-year-old age group) are less likely to become ill from certain diseases. The vaccinations which fall within the two-year-old group are for measles, mumps, rubella, diphtheria, tetanus, whooping cough, polio, hepatitis B, pneumococcus and haemophilus.

Consideration of immunisation rates gives a perspective on prevention that may not be provided from other indicators within our set; it also provides an indication of access to primary health care services.

What do the figures show?Increasing take up of all appropriate vaccinations has been the subject of a health target designed to promote uptake. The latest data suggest that rates are high (although the target of 95 percent by June 2012 had not been achieved by March) and that variation is relatively low. In fact 30 out of 35 PHOs reported rates in excess of 87 percent, representing a very small spread among the vast majority of PHOs.

Table 3: Age appropriate vaccinations for two-year-olds

AllNew Zealand 90%

Range across PHOs From 79% to 94%Source: PHO performance programme March 2012

What are the important considerations or potential caveats?The measure, as collected through the PHO Performance Programme, will include only those two-year-olds who are enrolled with a PHO.

5.9 Health care cost per capita (US$ Purchasing Power Parity per capita) 5.10 Health care expenditure as a proportion of GDP

What do these indicators measure?These two measures are different and complementary ways of considering what New Zealand is spending on its health care. Purchasing Power Parity (PPP) US$ is a way of saying absolutely how much is spent on health care per person using a technique to equalise the purchasing power of local currencies. In contrast, proportion of GDP spent on health care is a relative measure which shows how much of a country’s income goes to fund health services. Combined, the two allow us to consider both the cost and relative affordability of a country’s health system, and, when combined with quality measures, provides a starting point for considering the efficiency of the system.

Why are we interested?Like the rest of the developed world, expenditure on health services in New Zealand has risen dramatically in the last ten years: reflecting an ageing population and the development of medical technologies. This is leading all developed economies to question the extent to which such increases can continue, something recognised in the New Zealand Triple Aim. This trend has been exacerbated by global economic instability.

What do the figures show?

New Zealand’s position on these indicators is interesting. On the one hand, expenditure per capita is relatively low, with only accession countries from the former Soviet Bloc and a number of developing economies in the OECD list spending less per head on healthcare. However, as a proportion of GDP, expenditure in New Zealand is relatively high. What this implies is that while healthcare is relatively cheap in New Zealand by international standards (and its quality generally comparable with the rest of the developed world), New Zealand is less able than many to easily increase its health spending.

What are the important considerations or potential caveats?

There is no “right” level of expenditure for health care. It is certainly not the case that more expenditure will necessarily drive better outcomes, but equally, low spend does not necessarily equal greater efficiency

Figure 12: Health care cost per capita (US$ Purchasing Power Parity per capita)

United St

ates

Switze

rland

Luxembourg

Denmark

German

y

Belgium

Swed

en

United Kingd

om Italy

New Ze

aland

Israel

Slova

k Rep

ublic

Hungary

Estonia

Mexico

0

1000

2000

3000

4000

5000

6000

7000

8000

9000To

tal e

xpen

ditu

re o

n he

alth

care

/cap

ita, U

S$

purc

hasin

g po

wer

par

ity

Figure 13: Health care expenditure as a percentage of GDP

United St

ates

France

Denmark

Switze

rland

Belgium

Swed

enIce

land

Irelan

dSp

ain

Finlan

dChile

Israel

Hungary

Estonia

Mexico

0.0

2.0

4.0

6.0

8.0

10.0

12.0

14.0

16.0

18.0

20.0

Heal

th ca

re e

xpen

ditu

re a

s % o

f GDP

Source: http://www.oecd.org/document/16/0,3746,en_2649_37407_2085200_1_1_1_37407,00.html

6. Further indicator development

The next phase in developing indicators involves consulting with key stakeholder and consumer groups. The Expert Advisory Group will then work through the feedback provided by stakeholders, in conjunction with the Project Steering Group. This may result in changes to the proposed list of indicators and measures.

Following approval of any proposed changes from the Commission Board, an intensive programme of activities follows over the remainder of the calendar year to refine and test the indicator definitions, and to put in place processes for gathering data.

Further work is also underway to determine how we can best present and publish information to meet the needs of different audiences. This will conclude with the launch and publication of the first set of national indicators at the end of 2012.

We will review and develop our processes for sector consultation and communication, to ensure we provide regular updates on progress and opportunities for on-going dialogue with key stakeholders.

Appendix 1 – Expert Advisory Group

Organisation Confirmed members

Quality and safety manager from a DHB and/or Quality and Risk Management Network representatives

1. Cate Tyrer (Director Quality,Safety and Risk Wairarapa DHB); and/ or

2. Chris Eccleston (General Manager – Clinical Governance, South Canterbury District Health Board)

Secondary health clinicians 3. Colin Feek (Clinical Director, Medicines, Community & Cancer CCDHB)

4. Andrew Brant (CMO Waitemata DHB and Public Health Physician)

Disability sector representative 5. Paul Gibson (Commissioner for Disability Rights, Human Rights Commission) - from November 2011 to February 2012

6. Rhondda King (Group Manager, National Quality, National Services Purchasing, National Health Board) - from February 2012 to present

Consumer representative 7. Ernie Newman (National Health IT Board consumer panel)

Data analysis expert (primary and secondary data sets)

8. Susan Iverson (Karo Data Management- involved in HQMNZ EAG)

Academic representatives working in the field of quality improvement (covering primary and secondary care)

9. Prof. Tony Dowell (Head of Department Primary Health Care and General Practice, University of Otago); and

10. Prof. Jonathon Gray (Auckland University/Ko Awatea) (secondary care focus)

PHO Performance Programme 11. Serena Curtis-Lemuelu (PHO Performance Programme Manager)

DHB representative 12. Vicky Noble (Primary Health Care, Capital and Coast DHB, involved in HQMNZ EAG)

Nursing representative 13. Colette Breton (Nurse Consultant, Hutt Valley DHB)

Primary health clinician 14. Dr John Wellingham (Apollo Medical Centre)