Diabetic retinopathy screening NSF-based training

Principles of Screening

Tunde Peto

Head of Reading Centre

The eye in relation to other diseasesKey issues for discussion• Review past experiences related to eye diseases• Review common diseases that have an impact on eye health• Review blindness statistics and discuss the impact of blindness on

the community • Review principles and protocols of NSCLearning outcome• Understand the complexity of dealing with eye diseases • Understand the need for structured approach to eye health and

diseases• Understand the need for appropriate classification of diseases• Understand the principle for screening and able to interpret NSC

requirements

Quality assurance

Key issues for discussion• Review the need for quality control • Discuss the implication of quality control

on the Screening Programme Learning outcome• Understand the need for quality control • Able to implement quality control in the

Diabetic Retinopathy Screening Programme

The Tragedy of Avoidable Blindness

• World Health Organization: an estimated 180 million people worldwide are visually disabled and 40-45 million of them are blind.

• About 90 percent of the world's blind live in developing countries. 80% of the world's blindness is avoidable, preventable or treatable with available interventions. Cataract is responsible for nearly 50% of world blindness.

• Dr. Gro Bruntlandt, Director-General of WHO, notes that if current population growth and aging trends continue, the number of blind people in the world could double by 2020.

Major causes of world blindness

• Cataract

• Trachoma

• Glaucoma

• Xerophthalmia – Vitamin A!

• Onchocerciasis - Onchocerciasis is a systemic infestation by a nematode worm that causes severe ocular inflammation. Onchocerciasis is widely endemic in tropical Africa and in pockets of Latin America. It affects more than 20 million people and, in some communities, is responsible for blindness rates of up to 50 percent of the population.

• Leprosy - Leprosy is a generalized communicable disease that often affects the eye with corneal and intraocular inflammation.

Major causes of UK blindness• Age-related macular degeneration

• Glaucoma

• Inherited diseases

• Diabetes

• Cataract

• Optic nerve diseases

• Br J Ophthalmol. 2003 Oct;87(10):1201-4.   What is the cost of blindness? Meads C, Hyde C.

What is screening?• WHO definition: Screening is a public health service in

which members of a defined population, who do not necessarily perceive they are at risk of, or are already affected by a disease or its complications, are asked a question or offered a test, to identify those individuals who are more likely to be helped than harmed by further tests or treatment to reduce the risk of a disease or its complications.

• (National Screening Committee : www.nsc.nhs.uk)

Limitations of screening

• Examines apparently healthy people• People might have unrealistic expectations• Has the potential to save lives or improve quality of life

through early diagnosis of serious conditions, but the condition needs to be manageable/treatable

• Can reduce the risk of developing a condition or its complications but it cannot offer a guarantee of protection.

• There is an irreducible minimum of false positive results (wrongly reported as having the condition) and false negative results (wrongly reported as not having the condition).

When should we set up a screening programme for a disease?

• If you have a well defined population with a genuine health problem

• You must be able to define who is at risk and who is not

• You must have treatment available and it must be effective

• Screening also must be cost-effective

• It is unethical to screen if you cannot treat!

Aims of DR Screening • Primary aim is to diagnose, then refer and consequently

treat sight threatening DR retinopathy • Help the patient and health professionals to identify

further care issues based on level of DR • It also provides opportunistic diagnosis of cataract,

glaucoma, AMD and other eye diseases photographed, but it is not a definite aim of the screening.

• Patients with diabetes already have a disease, but this does not mean they do not find screening stressful (patients attending pap-smears, breast cancer screening believe they are healthy – major difference)

Why do we screen for diabetic retinopathy?

• Defined population who is at risk of blindness

• We have laser treatment available that is effective and cheap

• Diabetic retinopathy is “silent” for a long time, vision might remain good despite sight threatening disease

• Treatment is more effective if administered early in the disease, as it aims to stabilise vision

Will screening pick up all patients needing referral?

• Short answer: NO, it will not. Every screening has patients who have the disease and were deemed normal (false negatives) and who do not have the disease and were told they have it (false positive).

• The way to make sure that the least number of patients fall into these categories are continuous training and internal and external quality control. This can also reduce the risk of patients finding the screening too stressful.

Exeter standards

• These are for 80% for specificity and 95% for sensitivity

• How are they calculated? • Sensitivity refers to how good a test is at picking up disease• • True positives• ------------------------------ x 100• True positives + False negatives

• Specificity refers to the accuracy of the detection of images requiring referral or how good a test is at saying normal is normal

• True negatives• ----------------------------------- x 100• True negatives + False positives

Do you need patient’s consent? • Yes, you do. You have to provide enough

information that enables patient to make an informed choice. You must tell them honestly about process of screening, what will happen to the photographs, who might be contacting them if they need referral.

• Document reasons for non-consent and opt-out carefully. If possible, examine patient’s reasoning and answer concerns. Always consider if they might be having a hypo, especially if they are irrational!

Consequences of not enough information• Screening usually occurs in close-knit communities,

one bad patient experience might have serious repercussions

• Too stressful and the patient does not return• These in turn will have cost and target implications• If screening/grading is not right, it might result in over-

or underreferral, unnecessary cost to the health system; other patients missing out

• Patients who do not have the right information might become non-compliant, although life events, work, fear, denial can also play part in this

How to monitor for these?

• By using internal and external quality assurance

• Internal QA monitors everyday activities and compares them to local policies and protocols

• External QA compares a programme to others and to national standards; provides feedback to individual programmes, helps them to improve. All programmes help the national standards to be set and then identifies where they might be re-set.

Components of DR Screening

• Target population is identified and database created to enable admin staff to produce a reliable call-recall system, helping them to meet targets, monitor DNAs, refusals, and performance indicators via DR software

• Screening Test must be digital now, and every component of the test must have a standardised protocol that all screeners/graders follow (imaging, grading, referral, and monitor treatment targets)

• Administrative and IT support is essential

Who should be screened in England?

• All people with diabetes aged 12 years or older

• Special consideration should be given to housebound, nursing home and prison population

• Patients under special consideration should be on the database, and managed according to their abilities and local pathways

1: To reduce new blindness due to diabetic retinopathy

• Criteria: Annual blind registration, compared to 1990/1 rate of 9.5 per million per yr.

• Minimal standard: 10% reduction within 5 yr. Achievable standard: 40% reduction within 5 yr.

• Data held by: National database, but baseline data will need to established locally as well

• Relevance: will influence quality of life, reduce cost of treatment by providing early treatment

2: To identify and invite all eligible persons with known diabetes to attend for the DR

screening test

• Criteria: Completeness of database

• Minimal standard: Proportion of GPs/people on register: 90%

• Achievable standard: Proportion of GPs/people: 98%

• Percentage of patients invited: 100%

• Systematic call recall from single centre on a collated list

• All newly dg-d patients screened within 3 months

• Data held by: Various agencies, so collaboration is key; single collated list must be created

• Relevance: Less chance of patients not being screened or being lost

in the system

3: To ensure database is accurate

• Criteria: Accuracy of database of persons age 12 or more, as determined by Post Office returns

• Minimal standard: 95%• Achievable standard: 98%• Data held by: Screening database• Relevance: Number of patients screened

remains high; good internal QA needed to keep the database up to date

4: To maximise the number of invited persons accepting the test

• Criteria: Percentage of eligible persons accepting the test1. Initial screen: Min: 70%; Achievable: 90%2. Repeat screen: Min: 80%; Achievable: 95%

• Data held by: Screening database • Relevance: GPs need to be notified of non-

consenting patients, so they can follow it up; patient education might be given;

5: To ensure photographs are of adequate quality

• Criteria: Percentage ungradeable (Level U) • Minimal standard (M): cataract: 10%; no cat:

5%• Achievable standard (A): 5% and 3% • Data held by: Needs to be on the grading form as

a separate option • Relevance: Influenced by population statistics,

availability of ophthalmic services prior to DR Screening; Re-training of photographer if needed; ungradeable pathway must be used appropriately

6: To ensure grading is accurate

• Criteria Inter-grader agreement: M: 75-90% A: 85-95%Uncertain grading: M: 90% A: 95% Provision of external and internal QA activity• Data held by: Screening database for grader’s

identity; Certified graders only• Relevance: Reduce errors in referrals; Supervising

body to have a program ready for retraining

7: Ensure optimum workload, maintain expertise, avoid errors

• Criteria: • Graders: 1000–5000 pts read/yr• Optometrists/ophthalmologist: 500 pts read/yr• Fatigue: we found that after 4 (max 6 hours

occasionally) error rate increases • Data held-by: Quality control grading, monitoring

grading hours• Relevance: Reduce errors; provide internal and

external quality control

8: To ensure timely consultation of abnormal screening

• Criteria: Time between screen and grading for fast-track referrals (R3)

• Minimal standard: 95% referred within 1 week, 100% within 2 weeks

• Achievable standard: 98% referred within a week • Data held by: Date of referral is held by the DR

screening softare, Date of Clinic is held by hospital• Relevance: Clinic dates are to be tracked through the

ophthalmic care provider, so timely treatment can be provided to those most in need

9: Ensure both GP and patients are informed of all test results

• Criteria: Time before posting notification letters to GP and patient

• Minimum: 70% in less than 3 weeks

• Achievable: 100% in less than 6 weeks

• Data held by: Database

• Relevance: All parties concerned know about the outcome of the screening and can act on it appropriately while avoiding double referrals

10: Ensure timely consultation for all screen positive patients

• Criteria: Time between notification of positive test and consultation

• Maculopathy (M1) M: 70% A: 95% <13 weeks

• Preproliferative (R2) M: 70% A: 95%< 13 weeks

• Proliferative (R3) M: 70% A: 95% <2 weeks

• All grades: 100% less than 18 weeks

• Data held by: Clinic notes and hospital database

• Follow-up: Timely treatment reduces risk of further visual loss

11: To ensure timely treatment of those listed by ophthalmologist

• Criteria: Time between listing and laser1. Proliferative DR: M: 90% A: 95% <2wks 2. Maculopathy: M:70%; A: 95% <10wks

• Data held by: Date of Clinic and the date of laser are held by the ophthalmic care provider

• Relevance: Laser treatment is to stabilise vision and preserve sight, so timely treatment is more likely to help patient preserve the sight they have

12: To minimise overall delay between screening and first laser

• Criteria: Time between screening and treatment (if listed) does not exceed

• R3: M: 70% A: 95% <4 wks; 100% within 6 wks

• M1: M: 70% A: 95% <15; 100% <26 weeks

• Data held by: Hospital database/notes

• Relevance: Good communication between screening and clinics needs to be maintained; laser designed to preserve sight, not to give sight

13: To follow up screen-positive patients

• Criteria: DNA rate for ophthalmologist for

PDR within 1 month: M: 10% A: 5%

Maculopathy/R2 within 6 months: M: 10% A: 5%

• Data held by: Hospital notes

• Relevance: The F/U for these patients is standard hospital procedure, but information needs to go back to the Screening Programme to make sure that patients screened/or not on time

14: To minimise the anxiety associated with screening

• Criteria: Monitor false positive rate of DR test: M: 25%; A: 20%

• Data held by: These data are to be generated from the Quality Control grading and clinic notes

• Relevance: To have a procedure on retraining for staff and re-notifying these patients; ensure best use of resources

15: To ensure timely rescreening • Criteria: Time to rescreening compared to

suggested screening interval• M: 70% within 12 months• A: 95% within 15 months• Data held by: Screening database • Relevance: Ensures no patient is lost in the

system and have the maximum possibility to have their sight threatening disease identified

16: To ensure the public and health care professionals are informed at

regular intervals

• Criteria: Production of annual report

• Data held by: Screening database, Participating PCT and GPs and ophthalmic care providers

• Relevance: Identify the performance of every component of the screening and formulate plans to improve where needed

17: To ensure the service participates in quality assurance

• Criteria: External quality assurance• Minimum requirements:

– Evidence of participation of all graders in external image test set scheme

– Participation in peer-review visit programme– Annual submission of national minimum data set by 31st

October

• Data held by: Screening and clinical databases• Relevance: Identify areas of weaknesses and address

these by appropriate re-training

18: To optimise programme efficiency and ensure ability to assure quality of

service

• Criteria: Minimum programme size• Minimum standard: Population including

12000 people diagnosed with diabetes on current patient list

• Achievable standard: 15000 people on list• Data held by: Screening database• Relevance: All graders have enough experience

in all disease state; good use of resources

19: To ensure that screening and grading of retinal images are provided by a trained and competent workforce

• Criteria: Accreditation of screening and grading staff in accordance with national standards

• All staff should be accredited within 2 years of appt; current staff by April 2008

• Data held by: Screening services • Relevance: Patient protection; good quality

service overall

Progression to high risk proliferative diabetic retinopathy

Mild Moderate Severe V severe

1 yr 1% 3% 15% 45%

5 yrs 15% 27% 56% 71%

External quality assurance – what to expect and how to prepare for

it

Dr Tunde Peto

Head of Reading Centre, MEH

Based on NSC Draft Guidelines for external QA, courtesy of Fionna O’Leary

External QA• Organised nationally, led by the national QA

director and supported by the regional QA managers

• Will work closely with SHA • Regional QA will disseminate standards, share

good practice and raise issues of professional concern

• QA Manager will identify and act on areas that are felt to be operating at risk

• Accountable to National Programme Manager and National QA Director

Purpose of the QA visit

• Assure the quality of the DR screening by: – Assessing compliance with the national

standards, for all program parts and their interrelations

– Reviewing quality of data– Identify and encourage good practice– Identify areas for improvement and make

recommendations how to achieve this

Structure of the visit • Prior to the visit the Regional QA Manager

will ask to see the evidence for: – PCT commissioning – Processes and protocols in place for

administration, provision of patient information, photography administration of eyedrops, grading, referral and treatment

– Annual report for the programme – Completion of the pre-visit questionnaire sent by

the QA office

Visit components • PCT commissioning

• Call/recall administration

• Photography

• Grading

• Ophthalmoscopy

• Process of co-ordination of each components of the programme

There might be more than one visits to complete all elements of the QA visit.

Who will attend the visit? • Regional QA Manager will invite the

screening lead from the SHA Public Health Dept to attend

• Any or each of the following: – QA Director, National Programme Director,

screening commissioner, local DPH, administration manager, grading manager, data analyst, technical specialist, other members of the national team, GP, optometrist, patient representative, observers, trainee QA team members

Cycle of visits • QA visits aim to provide learning environment

• Visits will occur approx every 18 months

• Tour of the grading room, the administration centre

• Visit of the screening appointment and ophthalmology

• Review of working environment, facilities and equipment

• Meet the staff working in screening

Notification and duration of visits • Normally at least 4 months notice will be given

• Length of the visit will depend on the complexity of the programme

• Visits may take one day, but may be spread over a longer period (up to 2 weeks) to see satellite clinics

• Programme Manager needs to take responsibility for organising the visit locally

Key staff to include• Clinical lead/programme director• Lead ophthalmologist from the acute Trust• Grading room manager• Administration manager• Person responsible for maintaining equipment

and software issue log

Date for the visit will only need to be changed in exceptional circumstances

Programme manager’s duties

• Assist with practical arrangements for the day

• Trust need to provide suitable venue, including small room for team visit and one-to-one discussions

• Provide maps to all venues and organise car parking

Pre-visit questionnaires • National format and include a list of supporting

documents for the QA team to see before the visit • People who are in post and responsible for certain

elements of the screening need to fill in the questionnaires

• They will be sent at least 2 months in advance and need to be sent back at least 4 weeks before the visit

• Relevant members of the visiting team will have copies of the completed questionnaire

Conduct of a QA visit • May vary according the the QA team • If small team, probably 1-2 hours per segment of

screening is sufficient• Assessment of the role of the programme manager

and the clinical lead • Then a patient journey is followed• Review of photography and grading • QA Team will assess if feedback on the day is likely

or not; if yes, the Programme manager needs to book a room big enough to hold the team and the CEO-s, DH, commissioners, etc….

At the end of the visit,

• The QA team will reconvene in private with the QA director to consider recommendations and feedback

Programme elements to be reviewed

• One: Programme commissioning and public health– Pre-visit questionnaire to be completed– Appointment of clinical leadership, identification of

job description and identified session time – This should clarify what has been commissioned and

what is believed to have been commissioned and identify gaps and help with future planning

– Highly desirable for the Director of Commissioning and DPH to be present

Programme elements to be reviewed

• Two: Call/recall and administration– Annual report and pre-visit questionnaire provides the basics– But all supporting information needed, such as patient

information leaflets, letters of invitation, opt out and results letters, office manual and ANY other policies and protocols

– Links to GP practices and support from GP practices will be discussed and reviewed in order to encourage uptake in ethnic minorities

– Working practices such as database settings, searches, completeness of the database, methods and frequence of updating the database, management of exclusion, appts system, PO returns, referrals follow-up

– Programme Manager and admin team should attend

Programme elements to be reviewed

• Three: Screening and grading – Annual report, pre-visit questionnaire, information shown to

patients at screening appts, policies and protocols, equipment, working environment, workload, workflow, training and qualification will be reviewed

– Feedback from arbitration and from clinical lead discussed

– External Test Set (ETS) results are given: anonomysed, but clinical lead will distribute it to individuals

– Review a set of grading outcomes with the clinical lead

– Usually takes about 2 hours

Programme elements to be reviewed

• Four: Ophthalmology– Annual report, ETS, pre-visit questionnaire,

commissioning, processes to ensure that national standards are met, processes and protocols how feedback is given, assessment of ungradables and its QA, treatment of all people with diabetes, patient pathway referrals

– Electronic access will be assessed

– Leadership, content and attandence of multi-disciplinary team meeting and dissemination of information provision and reporting is discussed

– Links to diabetology department will be discussed

Feedback • If possible, verbal feedback will be given at the end of the

visit • Everyone taking part is invited • Visiting team will answer questions• Good practice will be identified and celebrated • Recommendations for service improvement will be given • Areas of particular concern will be identified so urgent

action can be taken• Personal criticism or warning will not be given in the

group feedback, but will be raised with the CEO

Report• Written report is to be provided to the programme within

4 weeks for validation, any errors should be highlighted within 2 weeks. No response will mean no error!

• Each programme part will be commented on• Main focus is compliance with standards• Named individuals will not feature in the report, but will

be sent to CEO • Clear recommendation for action and risks for service• If additional funding is required the commissioners will

be jointly responsible for acting on the report

Sharing the report• Copies will be sent to:

– Trust(s) involved– National Programme Director– DPH of the relevant SHA– Lead commissioner in PCT– Other relevant PCT leads

• Trust/PCT are expected to develop and action plan and provide a response in 4/12

• The report and any response will be published on the NSC website!

• QA documents are to be kept for 8 years

How to avoid ruining your own screening programme?

• You should know

– the disease and its complications

– the Screening or Study protocol: where you are based

– the photography and grading protocol

– the aims of screening: clinical pathway directions

– your collaborators: from GPs to consultants

– where to get help: phone numbers and clinic time

– the deadlines