Michael Gillin, PhD Professor, Chief of Clinical Services,

Dept. of Radiation Physics

Michael Gillin - Grandfather

My twin 12 month old grandsons, Arthur and Charles, live in Minneapolis.

UT MDACC > 8,000 New Patient Starts in FY 12

Quality Assurance The Dilemma

• It is simply not possible to insure safe and accurate treatments under all circumstances, but it is expected.

• The systems (TPS, EMR, Network, and treatment delivery) are too complex and are ever changing and responsibility is diverse.

• Comprehensive knowledge of the entire process is limited.

• Time is finite and there are multiple priorities. • What takes Physics so long to commission

versus what are the Physics concerns.

The Unexpected Finding: Would we observe it and understand its importance?

The Unexpected Finding: Would we observe it and understand its importance?

Quality Assurance Extra-ordinary Event

Rinecker Proton Center Event • Summary: 15 fractions delivered without

incident. Fraction 16: The entire prescribed dose was delivered in 1 fraction. This was not understood until the next day when the EMR indicated that the prescribed dose had been delivered.

• Could this happen to us? Can systems be established to guard against a unique set of circumstances? The New York Times.

Quality Assurance Extra-ordinary Event

Rinecker Proton Center Event Possible Lessons

• How many FTE’s should be dedicated to testing, monitoring and attempting to defeat systems, e.g. EMR, delivery, institution SW, ( Anderson filter)? UT MDACC ~ 1 to 2% of QMP’s

• How frequently should QMP’s perform end-to-end testing, i.e. image, plan, QA, and treat a phantom, in the Treatment Mode of the Delivery System? Annually per unique delivery system.

Quality Assurance Extra-ordinary Event

Rinecker Proton Center Event Possible Lessons

• Sensitivity – to general comments – efficient use of time?

• Staffing – Is sufficient priority given to reviewing to routine operations

• Training – In-service to review the entire delivery system – Vendor supplied

Radiation Oncology Systems Safe and Accurate

Multi-vendor, integrated, digital, complex delivery systems. Photons: UT MDACC is evolving from one standard

delivery system to different vendors and different systems from the same vendor (HDMLC, Rx couches, different in-room imaging.)

Protons: Scanning beams and scattered beams Comprehensive knowledge of the entire system is quite

limited. QA lags behind the new systems – slowly implementing

Atlas, a DB which is designed to store QA measurements ∞ Number of tasks, including various combinations Limited amount of time, which must be used wisely.

Radiotherapy Risk Profile Technical Manual World Health Organization Quality assurance (QA) in radiotherapy is all procedures that

ensure consistency of the medical prescription, and safe fulfillment of that prescription, as regards to the dose to the target volume, together with minimal dose to normal tissue, minimal exposure of personnel and adequate patient monitoring aimed at determining the end result of the treatment.

WHO 1988 MDACC averages 7 variances per month in the last 6 months,

zero reportable variances, i.e. 7 less than perfect events per month out of > 10,000 possibilities.

Radiation Oncology Systems Safe and Accurate

Informal Survey 12 Medical Physicists What are the most important safety questions for patient

care in radiation oncology at UT MDACC? Training – Human Error – Don’t know what we don’t

know Oncologists, RTTs, Service Techs, Physicists

Interconnectivity and Communication Stability of network, of applications (as SW changes),

multiple groups (Hospital, Rad Onc IT, Physics, Vendors) Limited time

Commissioning, Maintenance, After repair, Treatment Schedule

Unexpected Failure of Some Aspect of the Delivery System Unknown double failure

Reasonably prudent practice: Completely trust the vendor? Test in Treatment Mode

Did Varian get the control system right?

Introduction of New Systems – Education of Users and Vendors

Somewhere there is a linac. Reasonably prudent practice:

Understand the entire system

Vendors have work to do! Elekta has promised a better schematic.

The multi-vendor challenge. Vendor A points to vendor B who points to vendor C, who did not realize there was an issue.

ASTRO – U.S. Perspective

Radiation Oncology Treatment Delivery Systems

CT Simulators – 4 different models of CT simulators Treatment Planning Systems

Photons – Pinnacle Protons – Eclipse HDR (3), PDR (4), Prostate, Eye, etc.

Electronic Medical Record - Mosaiq Treatment Delivery Systems with in-room imaging

Varian linacs (16 units + Gamma Knife) Main Campus 8 in RCCs Hitachi Proton System (4 scattered beam lines + 1 spot beam line)

Digital systems do have internal monitoring functions. Emails from the delivery systems –TrueBeam or Hitachi

sending out emails stating the results of various internal tests?

What version of SW will the device alert the user that some parameter has changed? 2014, 2015, 2016 ?

G3 Scan Beam Weekly Output Checks**

2012 Data • Range Average SD • 30.6 cm 98.5% (1.0) 0.68% • 20.0 cm 93.1% (.94) 0.24%* • 14.0 cm 80.8% (.82) 0.50%*

• * Incorrect baseline data corrected in the

summer 2012 • ** Technique changed from 5/10 phantom to

Tracker Lack automatic processes to collect and analyze this data taken weekly. A great

need is to develop processes to automatically analyze QA data.

Clinical Physics Staffing UT MDACC

Clinical Qualified Medical Physicists - 36 Medical Physicists working towards certification – 6

Total ~42 Physicist Assistants

Main campus – 6 each PTC H - 5 each

Service Personnel Main campus – 8 PTC H - 5

There are other physicists whose primary responsibilities are research, education, RCCs, and RPCs.

Clinical Physics Staffing

~ 1.5 physicist per external beam delivery system PTC H is staffed by a QMP from 7 am until mid-night, 5

days per week PTC H is staffed every Saturday by several QMPs Photons – limited staffing on weekends – weekdays

staffing from 7 AM until 9 PM

~ 1.5 physics support person per external beam delivery system plus support from vendors (Varian, Hitachi, Elekta, etc.)

Clinical Physics Staffing

Estimated QMP Machine QA FTE per delivery system: Protons – 1 FTE per beamline for machine QA

G1 – 16 beams G2 – 23 beams G3 – 94 beams

Photons - 0.1 FTE per linac for machine QA TrueBeam – 3 photon beams and 5 electron beams

Multiple photon systems, one proton system.

IMRT QA Photons

This is basically just a physics service. We do not use this data to identify issues to be addressed.

MDACC Issues: Highly modulated fields, split fields, and couch kicks

IT Staffing Infrastructure IT, Application Physics

Radiation Oncology Mosaiq, InfoSec, Desktop

MDACC Network InfoSec

Vendor Elekta – one on-site person Eclipse – one on-site person

The division of responsibilities between medical devices, infrastructure support, and information security is evolving, as priorities and standards keep changing. There are very different cultures in IT and clinical physics.

Peer Review A Developing Discipline

Peer review within Physics Define expectation of independent physics review

‘Peer’ review within the Radiation Oncology practice Opportunity to review before a new

device/version/procedure is used Executive summary for CQI Committee who must

respond in some fashion.

RPC services

Recent QA Issues Error in Image Association, which appears to be a network

issue with time delays as data is sent to the Server. Confusion in the EMR definition of a fraction, with

potential patient underdose Failure to deliver the entire dose with spot scanning and

with x-rays Custom coding for electron apertures, two years to

implement, due to design and other issues (vendor support)

Appropriate integration of different systems, e.g. TrueBeams with 15 MV, HDMLC, 3 different types of treatment couches, and soon Elekta linacs into one practice.

Off-label use of device without any review

UT MDACC Standards Responsibility for Equipment Quality Control

“The Quality Control Officer is responsible for assuring that all equipment quality control procedures are adhered to, and that appropriate documentation is maintained. The Radiation Treatment Quality Assurance Committee monitors equipment quality control activities and indicator of equipment performance arising from quality control activities.”

Quality Assurance Guidance for Canadian Radiation Treatment Programs, April 3, 2011

TJC requires a Quality Officer for each clinical department, but with different responsibilities, 0.1 FTE.

UT MDACC Standards Quality Control Officer

“A qualified individual is designated as having primary responsibility for assuring that all equipment quality control procedures are adhered to, and that appropriate documentation is maintained. The Quality Control Officer reports to the Organization’s CEO either directly or through the Cancer Program or Organization’s quality committee or equivalent, on matters relating to quality assurance within the Radiation Treatment Program.”

Quality Assurance Guidance for Canadian Radiation Treatment Programs, April 3, 2011

Quality Officer reports independent of the radiation treatment program head. (In the US, radiation safety is structured this way.)

Institution Patient Safety Processes Historically patient safety processes in Radiation

Oncology have been independent of institution wide processes.

This will eventually change at UT MDACC, which is an important step, in my opinion.

Senior radiation oncologist “Mike, we trust you.”

UT MDACC Standards Equipment Quality Control Procedures

“Commissioning of equipment is independently reviewed, and checked with measurement as necessary, by a qualified individual, normally a medical physicist who was not involved in the commissioning process. Commissioning is not considered complete until this independent verification has been performed.” RPC TLD Report is required before the first treatment.

Quality Assurance Guidance for Canadian Radiation Treatment Programs, April 3, 2011

UT MDACC Standards Introduction of New Equipment

“Before new equipment is introduced into clinical service, a complete safety analysis is performed, a quality control procedure is implemented and tested and all personnel involved in the calibration, operation or maintenance of the device are trained in the operation of the device.”

Quality Assurance Guidance for Canadian Radiation Treatment Programs, April 3, 2011

MDACC performs an independent safety analysis and independent safety tests on each new piece of equipment before first clinical use.

Clinical physicists have not been trained to perform complete safety analysis, but are learning.

UT MDACC Standards External Dosimetry Audit

“An independent machine dosimetry audit, available from the Radiological Physics Center (RPC) in Houston, Texas, is conducted on an annual basis. The audit results are reviewed by the Radiation Treatment Program Head, the Radiation Quality Assurance Committee, and the heads of the radiation oncology, medical physics and radiation therapy departments.”

Quality Assurance Guidance for Canadian Radiation Treatment Programs, April 3, 2011

MDACC shares the results of these audits with the physics team leads.

RPC External Audit Results

6 MV Variation + 2% over 6 years

2108

6, 9, and 12 MeV Electrons

Variation + 3% over 6 years 2109

RPC External Audit Protons Spot Scanning Beam

In the middle of a 10 cm x 10 cm x 10 cm volume, maximum range 30.6 cm, uniform dose with 8 mm spot spacing, 1 cGy = 1 MU

TLD/Inst ratio 0.965 to 1.005

RPC Liver Phantom Spot Scanning

Eclipse PTV1 RPC vs Inst 0.93

PTV2 RPC vs Inst 0.95 Not the best agreement – what can be learned from this?

Quality Assurance in Treatment Delivery

• Routine (TG 40, TG 142) – Limited time by QMP – Machine: Daily, Weekly, Monthly, Annual – Patient: Pre-treatment review, end of treatment

review, weekly reviews – After repairs – improvement needed to define

minimum tests after specific repairs • Extra-ordinary – Highly specific issues – Time

consuming – Limited Vendor Support – Aborts during G3 treatments – EMR definition of a fraction

• Commissioning – New versions of SW – Eclipse 1 man year, Pinnacle

0.5 man year (multiple scripts, DICOM changes, etc.) – New systems of HW or SW – EMR multiple

disciplines – 4 to 6 months of testing

Role of QMP Routine QA • Machine

– Define measurements to be done – Analyze measurements - new systems

needed to track measurements. – Make measurements (PA and QMP)

• Patient – Pre-Rx chart review – changes several

percent. Important service – Post-RX chart review – Zero issues identified – Weekly reviews (CMD and QMP) –Not

important, especially with EMR.

MDACC IMRT QA 2012

2012 – performed 3035 IMRT QA Plans 98.7% passed ion chamber tests using a 5% criteria.

Oncologists expect high passing rates.

How valuable is this point dose test? How would oncologists react if physics designed tests

which had a 90 or 95% passing rate with the goal of identifying weaknesses in the system?

Should we be reporting dose to a point or dose to a volume to state the estimated minimum dose,

e.g. the CTV received 85% of the prescribed dose?

IMPT MFO Planning and Patient QA

» Chordoma – 34 yo female

• Three Fields: 70 Gy in 35 Fxs • Field Range(cm) SOBP(cm) Layers Spots MU • LAO 12.57 7.64 27 494 12.59 • RAO 11.97 7.16 33 595 15.89 • Vertex 15.03 5.42 16 637 20.16

Depth Dose Curves

Red diamonds are the Matrixx measurements in plastic water.

Error bars correspond to 2% and 2mm.

There are time constraints which limit the measurements to be made.

RAO FIELD Depth 5 cm

• Gamma analysis for field BRAPB (Gantry = 282o). Upper left pane: measured dose plane; lower left pane: calculated dose plane; upper right pane: isodose line-comparison; lower right pane: gamma index map, 99.5% passes for 2% and 2 mm criteria.

MDACC RPC Photon Phantoms 2011 - 2012 Activities

Lung – SBRT H&N – Segmental and VMAT Prostate – Segmental and VMAT Spine – Segmental Liver – Segmental

All passed national standards.

MDACC RPC Proton Phantoms

• Prostate – Scattered beam June, ’11 Passed – Spot scanned July, ’12 Passed

• Lung – Scattered beam May, ’11 Passed – Spot scanned Jan, ’12 Passed

• H&N – Scattered beam Not irradiated – Scanned beam May,’12 Passed both

SFO an MFO • Spine

– Scattered beam Oct, ’12 Failed retry Spring 13

– Scanned beam Not irradiated Need to do

Patient Safety and Technical QA Strengths

Well defined standards High quality delivery systems

Weaknesses Continuous training on all systems 1% per year of cost? Pay

the vendor to come annually. Complexity of systems and lack of analytic tools Disconnect between therapists and physicists/service – maintain

the schedule, which will keep patients happy vs. fix the unit Opportunities

Better interface with vendors Better systems for analyzing routine data

Threats Groups outside of Radiation Oncology Economic pressures to treat patients

Patient Safety and Technical QA Summary

Radiation Oncology – very complex systems

Regulations/Standards – addressing traditional issues

Limited tools for analyzing the QA results

Difficult to define appropriate tests for the entire process

Comfort in the established routines which provide expected results

Discomfort when different questions or issues are raised, e.g. reviewing risk issues with use of equipment or thinking in terms of dose to volume or robustness of plan.

Arthur and Charles: 10 months old,

Focused on the essentials.

Regulatory Environment In U.S. modest effort

Federal Source tracking FDA devices HIPPA Information Security

State of Texas Radioactive sources, specific limited QA requirements Devices – specific daily, weekly, annual requirements

Atlas summary RPC phantom list – protons and photons RPC output Proton daily check Photon daily check Changes in last 5 years – independent safety

review, exe summaries, greater reliance on RPC phantoms

Risk analysis

Recommendations – recent Processes – new EMR, new TPS, new delivery Environment of Safety

Scanning Beam Energy Absorber Measurements, TPS, EMR

Blank EA AL0 – large snout AM0 – medium snout AS0 – small snout

EA in aperture slot AL67 – large snout AM67 – medium snout AS67 – small snout

EA setting from Eclipse

Only 1 aperture slot is used

Any change in the delivery system, also involves changes in other systems.

Physics Patient Specific Services

External Beam SRS (gamma knife jail) and SBRT IMRT QA, before the first treatment Proton patient QA, especially spot scanning – 2 methods

of determining the MUs.

Brachytherapy Afterloading units, including IOHDR Prostate brachy, eye plaques, misc.