Co-Registration of Intracardiac Echocardiography (ICE) and Computed Tomography (CT) for the Purpose of

Guiding Structural Heart Disease Intervention

by

Jonathan E. Toma, MD, FRCPC

A thesis submitted in conformity with the requirements for the degree of Masters of Science Department of Medical Biophysics

University of Toronto

© Copyright by Jonathan E. Toma 2019

ii

ii

Co-Registration of Intracardiac Echocardiography (ICE) and

Computed Tomography (CT) for the Purpose of Guiding

Structural Heart Disease Interventions

Jonathan Toma

Masters of Science

Department of Medical Biophysics University of Toronto

2019

Abstract

We demonstrate in this work a system for tracking the position and orientation of an intracardiac

echocardiography (ICE) image within a CT volume for the purpose of improving imaging

guidance for cardiac interventions. The method begins with a user-defined registration step,

followed by subsequent automated local searches within the CT volume. The local searches are

performed by a series of individual 2D-2D co-registration steps over various positions and

orientations within the 3D space using an edge-sensitive algorithm. A stepwise sampling method

is used to reduce the total number of processing steps. Validation testing using a silicone cardiac

phantom over 10 pullback positions demonstrated average position and orientation localization

errors of 4.1 mm and 8.1 degrees respectively. A high degree of rotation symmetry in the model

likely contributed significantly to the orientation accuracy results. Further work is required to

reduce computation time for the algorithm, including the use of parallel processing.

iii

iii

Acknowledgments

I would like to extend my heartfelt appreciation to my supervisors Drs. Graham Wright and

Brian Courtney for their support in this thesis work. This work would not be possible without

their ongoing support and guidance. Thank you to the additional members of my Masters

committee, Drs. Anne Martel and David Goertz for their advice throughout the Masters process.

Thank you to the members of Dr. Graham Wright’s lab, including Labonny Biswas and

Sabastian Ferguson for their advice and support regarding software development. Thank you to

the members of Brian Courtney’s lab, including Dr. Natasha Alvez-Kotsev and Dr. Jill Weyers

for supporting experiments, data collection, and providing insights with respect to data

interpretation.

Thank you to the employees of Conavi Medical Inc, including Bogdan Neagu for providing

experimental catheters and phantoms, and providing advice regarding experimental setup.

Thank you to the administrative support personnel from Dr. Wright’s lab and the Department of

Medical Biophysics, including Kimberly Allan, Donna Marie Pow, and Merle Casci for their

advice and guidance throughout the Masters process.

Thank you to Professor Jan Modersitzki and his group at the Institute for Mathematics and Image

Computing from the University of Lübeck their work in developing a suite of registration tools

for MATLAB used in our work.

Finally, I would like to thank most my wife Stephanie, and my children Audrey and Benjamin

(both of whom were born during this degree) for their continuing love and support.

iv

iv

Table of Contents

Chapter 1: Introduction and Background .................................................................................. 1

1.1 Clinical Background .......................................................................................................... 1

1.1.1 Structural Heart Disease Intervention ...................................................................... 1

1.1.2 Aortic Stenosis ......................................................................................................... 4

1.1.3 Surgical Aortic Valve Replacement (SAVR) and Transaortic Valve Implantation (TAVI) ................................................................................................ 6

1.1.4 Evolving Role of Imaging Guidance in TAVI ........................................................ 8

1.1.5 Case for the Use of Intracardiac Echocardiography (ICE) during TAVI .............. 14

1.2 Technical Background .................................................................................................... 17

1.2.1 Medical Image Co-Registration ............................................................................. 18

1.2.2 Search Method within the CT Volume .................................................................. 29

1.2.3 Prior Work in Cardiac Ultrasound-CT/MR Co-Registration for Use in Procedural Navigation and as an Adjunct to Magnetic Tracking Systems ........... 31

Chapter 2: ICE-CT Co-Registration for the Purpose of Guiding Cardiovascular Interventions ................................................................................................................................ 33

2.1 Introduction ..................................................................................................................... 33

2.2 Methods ............................................................................................................................ 34

2.2.1 Software environment ............................................................................................... 34

2.2.2 Process Overview ..................................................................................................... 35

2.2.3 2D-2D Image Co-Registration .................................................................................. 37

2.2.4 Search Method within the CT volume ...................................................................... 39

2.3 Experimental Setup ......................................................................................................... 40

2.4 Results ............................................................................................................................... 41

2.4.1 2D-2D Co-Registration ............................................................................................. 42

v

v

2.4.2 Search Method within CT Volume ........................................................................... 43

2.5 Discussion .......................................................................................................................... 47

2.6 Conclusion ......................................................................................................................... 49

Chapter 3: Summary and Future Directions ............................................................................ 51

References ..................................................................................................................................... 61

1

1

Chapter 1

Introduction and Background

The purpose of our work is to use software co-registration methods to automatically determine

the position and orientation of an intracardiac echocardiography (ICE) imaging surface within a

three-dimensional CT volume. ICE is a catheter-based ultrasound technique where high-

resolution B-mode ultrasound images and Doppler information are obtained from within the

heart and displayed to the operator in real-time to help guide minimally invasive procedures. One

major drawback of the technique is the difficulty in recognizing the orientation of the imaging

surface relative to three-dimensional (3D) cardiac anatomy from the ICE images alone. A

software algorithm capable of demonstrating to the user the 3D anatomic context of the ICE

imaging surface would therefore provide a significant advantage to the operator. The algorithm

developed in our work is intended for novel applications of ICE imaging, including transaortic

valve implantation (TAVI), which is a minimally invasive method to implant an aortic valve

bioprosthesis in patients with severe aortic valve disease. It is vitally important in this procedure

to understand the location of the current imaging surface relative to neighboring critical

structures, such as the aortic wall, coronary ostia (origins), and the plane of the aortic annulus. In

the upcoming sections, we will review the clinical indications, rationale for the use of ICE in

TAVI, medical imaging co-registration, and how co-registration techniques may be used to

determine the location of a two-dimensional surface within a three-dimensional volume.

1.1 Clinical Background

1.1.1 Structural Heart Disease Intervention

Structural heart disease (SHD) intervention represents a rapidly growing sub-specialty within the

field of interventional cardiology.1–3 Physicians practicing SHD intervention use minimally

invasive techniques to repair pathologies affecting large structures of the heart, such as

narrowing or leakage of heart valves, or anomalous connections between cardiac chambers.3

These procedures are commonly performed using device implants through a percutaneous route,

2

2

which generally describes the method for accessing deeper structures in the body through the

skin without the use of a large surgical incision. Percutaneous access in the field of structural

heart disease intervention is most commonly performed by entering a blood vessel using a

needle-and-wire technique (Seldinger technique), and subsequently using this access to pass a

catheter through the blood vessel to the area of interest.4 A medical device is then passed through

the catheter to its destination where it may perform its intended action. This technique has been

adopted from earlier work in coronary and peripheral angioplasty; however, there are several

unique considerations in the structural heart disease field, including the larger vessel diameters

required for the passage of equipment, greater importance of properly sealing the port of entry,

and the risk of damaging (or causing embolization of) adjacent tissues or critical structures.1–3,5

Another unique consideration in this field is the role of adequate imaging guidance, given the

soft tissue composition and mobility of the target pathology, the size of the equipment typically

used in these procedures, and the proximity to adjacent critical structures.6–8 These procedures

are typically performed in an x-ray fluoroscopy suite, or cardiac catheterization laboratory,

which does not routinely make use of imaging modalities with soft-tissue contrast, such as

ultrasound. When soft tissue contrast is required, such as when performing transcatheter

(catheter-based) mitral valve repair, imaging guidance is generally performed using

transesophageal echocardiography (TEE).9–11 During TEE guidance, an imaging probe is placed

in the lower esophagus via the patient’s mouth, and usually requires administration of general

anaesthesia for the patient to tolerate this technique for extended periods of time (Figure

1.1A).6,7,9,12 The main benefit of this ultrasound imaging method is the vastly superior spatial and

temporal resolution of TEE (Figure 1.1B) compared to its surface counterpart, transthoracic

echocardiography (TTE). In the lower esophagus there is less tissue separation between the

probe and the left side of the heart as compared to TTE, as well as fewer tissue and air-tissue

interfaces producing ultrasound scatter between the probe and target area of interest.12 Later in

this introduction, we will discuss the role of intracardiac echocardiography (ICE), an emerging

catheter-based technique for providing ultrasound-based soft-tissue contrast during SHD

intervention, without the need for general anaesthesia.

3

3

Figure 1.1: Transesophageal Echocardiography is performed by inserting the imaging probe into

the lower esophagus via the mouth (A). There is far less tissue separation between the imaging

probe and the heart using this method. TEE is able to produce imaging with vastly superior

spatial and temporal resolution compared to transthoracic echocardiography (B). Source:

National Heart, Lung, and Blood Institute; National Institutes of Health; U.S. Department of

Health and Human Services.

Computed tomography (CT) represents a widely used imaging tool for procedural planning in

SHD intervention.6–8,13,14 Most contemporary procedures in this field benefit from the prior

knowledge of three-dimensional anatomy derived from CT. An operator can perform detailed

geometric measurements regarding the procedural approach, as well as the size and shape of the

area of interest within the heart, which often is used in determining the size of the prosthesis

required.6,14 Specialized software may help inform the operator of the fluoroscopic angle on the

live procedural x-ray system required to accomplish a specific task without the use of live soft-

tissue contrast.15,16 For example, as a result of CT procedural planning, the use of TEE during

transcatheter aortic valve intervention dropped by almost half (47%) in a large French registry of

patients undergoing aortic valve intervention in a relatively short period of time between 2010

and 2014.17

4

4

One of the particular strengths of using CT-based imaging for procedural guidance is the three-

dimensional anatomic context provided, particularly with respect to adjacent critical

structures.6,7,13,14,16 For example, when performing aortic valve intervention, operators are able to

understand the distance from the aortic valve to the origins of the coronary arteries, as well as the

length of the valve leaflets that will be displaced in their direction in order to accurately assess

the risk of coronary artery occlusion during the procedure (a complication associated with both a

high rate of morbidity and mortality). The information from CT may either inform the operator

of patients who are at prohibitive risk of undergoing such interventions due to risk of coronary

occlusion, or may dictate an alternative strategy to addressing the risk, such as stenting the

coronary origin(s).18–20

1.1.2 Aortic Stenosis

Aortic stenosis is a life-threatening clinical condition defined by narrowing of the aortic valve,

which is most commonly caused by chronic buildup of calcium on the surface of the valve

leaflets, and leaflet fibrosis (scarring, Figure 1.2).21–23 It is a condition predominantly affecting

the elderly, and it is estimated that one in eight persons over the age of 75 has aortic stenosis in

its moderate or severe form.24 There are, however, genetic conditions predisposing subjects to

development of aortic stenosis at earlier ages, such as bicuspid aortic valve disease, which affects

1-2% of the general population,25 but represents as high as 50% of aortic stenosis cases at a mean

age of 70 years old referred for surgical management.26 Patients suffering from severe aortic

stenosis can experience symptoms such as dyspnea (shortness of breath), angina (chest pain),

syncope (loss of consciousness), or even sudden death due to the condition. The average length

of time between the onset of symptoms due severe aortic stenosis and eventual death is

unfortunately only 1-3 years,27,28 which represents a worse prognosis than most forms of

cancer.29

5

5

Figure 1.2: Progression of calcific aortic stenosis from a normal aortic valve (far left), to aortic

valve sclerosis (calcium present, but no evidence obstruction), to mild-moderate and severe

aortic stenosis (right).30 Reproduced with permission from Otto, E. M. Calcific aortic stenosis –

time to look more closely at the valve. N. Engl. J. Med. 359, 1395-1398 (2008), Copyright

Massachusetts Medical Society.

It is possible for aortic stenosis to be detected as an asymptomatic heart murmur during a milder

phase, which then may be followed conservatively over time until the development of

symptoms.31 Some cases of severe symptomatic aortic stenosis arise incidentally when a patient

is evaluated in the clinic or emergency setting for new-onset dyspnea, angina, or syncope. The

standard of evaluation of the severity of aortic stenosis is transthoracic echocardiography (TTE).

Using TTE, physicians measure the relatively higher velocity of blood travelling through the

narrow aortic valve orifice to estimate the gradient (pressure difference) across the aortic valve,

and estimate an aortic valve area using the principles of the Bernoulli equation.32 Aortic stenosis

is considered severe if the peak velocity across the aortic valve exceeds 400 cm/s, the mean

gradient (based on the average of the Doppler velocity profile) exceeds 40 mmHg, or the

estimated aortic valve area is under 1.0 cm2.33 A physician following a patient with aortic

stenosis will typically perform routine TTE studies to follow the progression of the disease, and

will seek definitive intervention (surgery or TAVI) for the patient if the measurements suggest

severe aortic stenosis in the presence of symptoms. Other reasons to proceed to intervention in

patients who have minimal or no symptoms include weakness of the heart muscle (depressed

ejection fraction), another reason for heart surgery (such as the need for coronary bypass), or

very severe pathology indicated by measurements (velocity greater than 500 cm/second, mean

gradient greater than 50 mmHg, or valve area less than 0.8 cm2).33 Depressed ejection fraction

and very severe aortic stenosis are acted upon earlier despite the presence or lack of symptoms

due to the significantly higher risk of sudden cardiac death.34

6

6

1.1.3 Surgical Aortic Valve Replacement (SAVR) and Transaortic Valve Implantation (TAVI)

Traditional open-heart surgery in the presence of severe aortic valve disease has been performed

in clinical practice since the 1950’s.35 Surgical aortic valve replacement (SAVR) is performed by

a cardiac surgeon in an operating room, and typically requires a sternotomy (opening of the chest

by splitting the sternum), cardioplegia (stopping of the heart), and a bypass pump circuit in order

to circulate blood during the operation. During AVR, the new valve sewn in the aortic position

may be either metallic (mechanical), or may consist of a bovine or porcine pericardial patch

attached to a metallic support frame (bioprosthetic). Mechanical valves are known to be more

durable than bioprosthetic valves over the long-term, however require anticoagulation (blood

thinning) at a high threshold over a patient’s lifetime to prevent valve thrombosis (abnormal

clotting of the valve). Bioprosthetic valves are more similar in form and function to a native

aortic valve, and will last on average for a period of 10-15 years before suffering from severe

structural degeneration, causing either severe aortic insufficiency (valve leak), stenosis, or a mix

of stenosis and regurgitation requiring re-intervention.36,37 Patients spend an average of 9 days in

hospital following SAVR,38 and it is associated with a mortality rate as low as 1.4% in low-risk

patients;39 however, this number can be substantially higher in elderly patients or those with

significant co-morbidities.40 The current accepted “gold standard” for assessment of operative

risk of death either in-hospital, or at 30 days, is the Society of Thoracic Surgeons (STS) risk

score. Patients with an operative mortality of <4% are considered “low-risk”, 4-8% are

considered moderate risk, and >8% are considered “high-risk” for complications from cardiac

surgery.41 Before the widespread use of TAVI, patients who were either deemed high risk or

inoperable for traditional cardiac surgery were often palliated when their aortic valve disease

progressed to the point of causing repeat hospitalizations for refractory heart failure and eventual

death.27

TAVI is a catheter-based technique for implanting a bioprosthetic aortic valve for patients with

severe aortic valve disease, without requiring an open-chest operation (Figure 1.3). The

procedure is typically performed either through the femoral artery – transfemoral approach, or

directly through the tip of the heart using a “keyhole” incision – transapical approach. The safety

of the transfemoral over transapical approach has been well proven in recent years, and is thus

the preferred method of access for most TAVI operators, unless severe femoral or iliac vessel

7

7

disease is present.42–44 The new valve is advanced to the position of the diseased aortic valve in a

collapsed, or “crimped”, form mounted on a catheter until deployed. TAVI valves are expanded

to their natural size and shape either by balloon inflation (balloon-expandable valve), or by the

outward pressure of the valve frame once unsheathed (self-expanding valve). The metallic valve

support frame of the TAVI prosthesis holds the diseased native valve permanently open, and the

bioprosthetic leaflets mounted in the centre of the frame begin working immediately as a

normally functioning aortic valve.

Figure 1.3: Transaortic valve implantation (TAVI) is a non-surgical method to place an aortic

bioprosthesis within a diseased aortic valve (left), most often due to aortic stenosis (top right).

The frame of the bioprosthesis is held in place by the old diseased valve, which it displaces

towards the aortic root wall (bottom right).43 Reproduced with permission from Smith, C. R. et.

8

8

al. Transcatheter versus surgical aortic-valve replacement in high-risk patients. N. Engl. J. Med.

364, 2187-2198 (2011), Copyright Massachusetts Medical Society.

Several clinical trials have demonstrated the benefit of performing TAVI for symptomatic severe

aortic stenosis in selected patient groups; however, the indication for the procedure and group of

eligible patients continues to expand. The PARTNER 1B trial published in 2010 demonstrated a

significant mortality benefit of performing TAVI compared to medical therapy alone for patients

with symptomatic severe aortic stenosis who are deemed not suitable candidates for open

surgery.45 The PARTNER 1A trial published in 2011 demonstrated the non-inferiority of TAVI

compared to SAVR for both safety and efficacy in patients deemed high risk for surgery

(determined by cardiac surgeon using guideline of STS risk score > 10%).43 The transcatheter

patients were at slightly higher risk of stroke and vascular complications; however, the surgical

group was at higher risk of major bleeding and new-onset atrial fibrillation. The PARTNER II

trial published in 2016 demonstrated non-inferiority of TAVI compared to SAVR for various

measures of safety and efficacy in patients deemed moderate risk for traditional surgery (STS

risk score 4-8%).42 The transfemoral-access cohort of the study actually had lower rates of death

or disabling stroke at 2 years compared to open surgery. The PARTNER III trial was recently

published in March 2019, which demonstrated superiority of TAVI via transfemoral access

compared to traditional open surgery over one year in low-risk patients with respect to a

composite outcome of death from any cause, stroke, or rehospitalization.46 Secondary endpoints

also demonstrated significantly lower rates of stroke, new-onset atrial fibrillation, and shorter

index hospitalizations in the TAVI group compared to surgery. Although the PARTNER series

of trials were performed using exclusively the Edwards Sapien™ family of TAVI bioprostheses,

the U.S. COREVALVE High Risk Study,47 SURTAVI,48 REPRISE III,49 Evolut Low Risk

Trial,50 and many other trials have demonstrated similar results for various competing TAVI

valve systems.

1.1.4 Evolving Role of Imaging Guidance in TAVI

Given the minimally invasive nature of TAVI, performing the procedure safely and effectively

relies heavily on the appropriate use of imaging guidance.6–8,14,15,19 As previously mentioned,

TAVI is performed in an x-ray fluoroscopy suite, either a routine cardiac catheterization

9

9

laboratory (also known as a ‘cath lab’), or a hybrid cath lab-operating room. In the earlier days of

TAVI implementation, the majority of patients underwent a planning transesophageal

echocardiogram (TEE), cardiac catheterization, and an aortic-phase contrast CT of the vessels

extending from the aortic arch to the level below the femoral artery bifurcations. The purpose of

the planning TEE was to assess the aortic annulus diameter, distance from the annulus to the

coronary ostia (coronary origins), leaflet lengths, and assess the degree of calcification of the

valve, aortic annulus, and the left ventricular outflow tract (LVOT).6 The annulus diameter from

TEE was used in conjunction with the measurements from CT to determine the size of the

prosthesis required. The amount of calcification at the level of the aortic annulus and the LVOT

helped predict the difficulty in expanding the valve, the risk of causing aortic rupture (an almost

certainly fatal complication), and paravalvular leak (leak around the valve frame) due to irregular

depositions of calcium. The distance from the level of the annulus to the coronary ostia, along

with the leaflet length allowed operators to assess the risk of occluding one or both coronaries

following valve deployment.6,7,13 As mentioned previously, a large French registry of TAVI

patients had demonstrated the rates of planning TEE on average from 2010 to 2014 had dropped

by 48%, and likely has continued to drop in contemporary practice due to the advances in

multidetector CT.17

Contemporary multislice, gated CT technology has allowed CT imaging to effectively replace

TEE for the above measures, and its widespread use for anatomic pre-procedural assessment has

changed the algorithm for assessment for TAVI eligibility substantially.6,14,16,19 A typical TAVI

planning CT study will include measures of aortic annulus diameter and perimeter at a defined

phase of the cardiac cycle, sinus of valsalva diameters (base of the cusps), sinotubular junction

(where cusps meet aorta), coronary ostia distances from the aortic annulus, and diameters of all

vessels extending from the access site to the valve (Figure 1.4).14,16 CT has been demonstrated to

more accurately predict post-implant paravalvular leak compared to TEE, given the shape of the

annulus is slightly more often oval than circular.51 CT images may be easily re-formatted in the

plane of the aortic annulus in post-processing, whereas TEE relies on the skill of the operator

during the study to correctly align with the plane of the annulus. The aortic annulus mean

diameter (at the level of the virtual basal ring, Figure 1.4) is used more commonly for sizing of

balloon-expandable aortic valves given the outward pressure has a tendency to ‘circularize’ the

annulus, whereas self-expanding valves are sized to perimeter at the same level, due to their

10

10

tendency to conform more to the pre-existing annulus shape. The pattern of calcium at the level

of the annulus and the LVOT play a particularly important role in predicting where paravlavular

leak may occur.52,53 Asymmetric calcium patterns in the sub-annular region or the LVOT

(especially towards the left coronary cusp) have been particularly predictive of aortic rupture

when using balloon-expandable valves, and therefore patients with this pattern often have their

procedures performed using self-expanding prostheses.54

Figure 1.4: Three-dimensional (3D) representation of aortic valve cusp layout. 3D arrangement

of the three aortic valve cusps is shown in magenta. Virtual basal ring (purple) represents the

circular plane in contact with the base of all three cusps (primary measure for TAVI valve

sizing). Anatomic ventriculo-arterial junction (light blue) represents the plane dividing

ventricular and aortic ‘compartments’ when aortic valve is closed. Sinotubular junction (darker

blue) represents the plane where the cusps transition into the tubular portion of the aorta.6

Reproduced with permission Bleakley, C. & Monaghan, M. J. The pivotal role of imaging in

TAVR procedures. Curr. Cardiol. Rep. 20, 9 (2018).

Specialized image processing software has allowed TAVI operators to estimate the optimal

fluoroscopic angle for valve deployment, orthogonal to the plane of the aortic annulus, using the

pre-procedure CT study.14,15 For fluoroscopy, the goal for the operator is to produce a view such

that the bases of the three aortic cusps are linear, visible, and equally spaced (Figure 1.5). This

represents a view perpendicular to the aortic annulus with greatest separation between the left

and non-coronary cusp.55 A virtual fluoroscopic view may be generated in the image processing

software based on the CT anatomy, which informs the operator of the optimal angle prior to the

start of the procedure.

11

11

Figure 1.5: Representation of ideal alignment of left (L), right (R), and non-coronary (N) cusps

(left image) by aortic root angiography (right image).55 Reproduced with permission from Kasel,

A. M. et. al. Fluoroscopy-guided aortic root imaging for TAVR. JACC Cardiovas. Imaging. 6,

274-275 (2013).

The particular value of a fluoroscopic angle orthogonal to the plane of the annulus is in

estimating the depth of the TAVI prosthesis implant upon deployment.56 Given the fluoroscopic

image is a 2D projection of the TAVI prosthesis passing through the ‘imaginary’ plane of the

aortic annulus, any projection off-axis will significantly either over-, or underestimate the valve

implant depth. For example, the optimal implant depth of a CoreValve Evolut TAVI

bioprosthesis is between 2 and 6 mm from the base of the valve frame to the plane of the aortic

annulus. If the frame is lower (i.e. distance is greater than 6mm), the patient is at significant risk

of requiring a permanent pacemaker due to iatrogenic heart block. This potential complication is

due to the outward pressure of the valve frame on the adjacent AV node, or bundle branch fibers

in the interventricular septum. If the frame is higher (ie. distance is less than 2 mm), there is a

significant risk of paravalvular leak due to an ‘incomplete seal’ between the valve frame and the

native valve, and a non-trivial risk of valve embolization (valve frame moving completely out of

the plane of the annulus).52,56 If there is significant paravalvular leak due to an incomplete seal, it

is often required to place a second valve within the first valve at a lower position immediately

after the first one is placed to correct the problem. If valve embolization occurs at the time of the

procedure, the operator can use a snare to move the valve around the aortic arch to the level of

the descending aorta, and place a new TAVI valve in the correct position.57 The above

circumstances involve significant periods of hemodynamic instability prior to the placement of

the second valve (assuming the patient survives the event); furthermore, the placement of a

12

12

second valve predisposes the patient to a higher rate of long-term complications, including valve

thrombosis. For these reasons, it is of paramount importance to achieve the correct fluoroscopic

angle perpendicular to the plane of the aortic annulus, which is greatly informed by the pre-

operative CT.

The technique of using fluoroscopy as the sole imaging modality for guiding valve deployment

during a TAVI procedure represents the majority of cases in contemporary practice due to the

wealth of information provided by the planning CT study; however, a subset of operators

continues to use live TEE for selected cases for important reasons outlined below. As mentioned

earlier, TEE is performed by introducing an ultrasound imaging probe into the lower esophagus

via the patients’ mouth. TEE provides substantially better spatial and temporal resolution

compared to TTE due to the close proximity of the imaging probe to the left side of the heart,

and fewer tissue or air-tissue interfaces between the probe and the area of interest.12 In order to

perform TEE imaging during a TAVI procedure, a patient must be placed under general

anesthesia to tolerate the placement of the TEE probe for an extended period of time, compared

to the option of heavy conscious sedation, which may be employed in cases using fluoroscopy

alone.6,7,11,13 The use of general anesthesia requires heavier sedation, and involves placement of

an endotracheal tube to support the patient’s breathing throughout the procedure. Heavy

conscious sedation involves administration of significant amounts of sedation under the

supervision of an anesthesiologist; however, the patient breathes spontaneously throughout the

procedure. Patients under heavy conscious sedation compared to those under general anesthesia

are less prone to hemodynamic instability requiring the use of vasopressors to support their blood

pressure throughout the case.

Despite the drawbacks of general anesthesia, TEE continues to be used for imaging guidance in

selected cases due to the high-resolution soft-tissue contrast and real-time hemodynamic

information provided to the operator. A few definite advantages of intra-procedural TEE imaging

are: 1) the ability to determine the depth of implant in real-time without the use of contrast prior

to valve deployment; 2) the ability to assess the degree of paravalvular leak post-valve

deployment more accurately and without the use of contrast; and 3) the ability to monitor the

opening of the native valve leaflets in real-time during valve deployment, especially their

position with respect to the coronary ostia.11,13 For these reasons, intraprocedural TEE imaging is

particularly useful in cases where the operator would seek to limit exposure to contrast (such as

13

13

for patients with significant renal insufficiency), for patients with irregular patterns of calcium at

the level of the annulus or LVOT who may be more prone to paravalvular leak, or for patients

with low coronary ostia height and/or long native leaflet length who may be at higher risk for

coronary occlusion post-valve deployment. The late consequences of paravalvular leak post-

TAVI have been well-studied, and a published review of literature recently highlighted that it is

associated with a 2-fold increase in mortality at 2 years compared to trace or no paravalvular

aortic insufficiency at time of discharge from hospital.53

The issue of paravalvular leak is particularly important when attempting to perform TAVI in the

population with bicuspid aortic valve disease. As mentioned earlier, bicuspid aortic valve disease

represents as high as 50% of the aortic valve disease cases in patients at a mean age of 70 years

old referred for surgical management.26 The predominantly genetic condition causes the patient

to have two, rather than three aortic valve leaflets/cusps. These patients are prone to developing

aggressive forms of aortic stenosis at an earlier age.25 A subset of these patients is either

unsuitable or unwilling to under traditional open cardiac surgery and thus must to be considered

for definitive management with TAVI. The main issue arising in this group is the predominantly

ovoid shape of the aortic annulus (far less circular than the traditional tricuspid aortic valve

population) predisposing them to paravalvular leak. The availability of live Doppler for

paravalvular leak assessment afforded by TEE is therefore highly advantageous in performing

TAVI for these patients. The operator is better informed during the procedure regarding the exact

depth of implant and the amount of post-dilation of the valve required to more effectively reduce

paravalvular leak compared with angiography alone.58

Although there are several important advantages listed above of performing TEE during TAVI,

balancing these risks against the risk and logistics required for placing a patient under general

anesthesia continues to favor not performing TEE. Unfortunately, fewer patients are benefiting

from the real-time soft tissue and hemodynamic information obtained by TEE as the popularity

of x-ray-alone procedures continues to grow. The pressure from hospital administrations to

perform these procedures under conscious sedation is heavily influenced by reducing costs of

these procedures and associated length of stay in hospital.

14

14

1.1.5 Case for the Use of Intracardiac Echocardiography (ICE) during TAVI

Intracardiac echocardiography (ICE) is a catheter-based ultrasound imaging technique that

provides high spatial and temporal resolution images of the anatomy within the heart

surrounding the catheter. An ICE catheter is introduced via a venous approach (either femoral or

jugular vein) to the right side of the heart, where it is able to visualize nearby cardiac structures

with resolution similar to TEE. The ICE catheter contains a handle component that allows the

operator to steer the tip of the catheter and change the orientation of the imaging plane without

the help of a dedicated sonographer, which represents a significant advantage of this technique.

The main disadvantage of performing imaging from this perspective is the learning curve

required to understand the unique views inherent in ICE imaging. Users tend to overcome this

limitation with further experience; however it would be beneficial for the operator to have the

position and orientation of the ICE catheter displayed with 3D anatomic context, which is the

main purpose of our work.

Contemporary ICE imaging catheters can provide an imaging depth of penetration of up to 15

cm, which is adequate to visualize any structure within the heart. There are two main types of

ICE catheters available: 1) phased-array catheters, which provide a 90-degree sector view, and 2)

mechanical rotating element catheters, which provide a 360-degree view. A phased-array

imaging catheter, such as the Acuson Acunav ICE imaging system (Siemens, Mountain View,

CA, USA), uses a 64-element transducer with frequencies between 5 and 10 MHz to produce a

90-degree sector image (Figure 1.6A). A mechanical rotating transducer, such as the Conavi

Foresight ICE catheter (Conavi Medical, Toronto, ON, Canada) uses a single rotating imaging

element operating at 9MHz to produce a 360-degree view (Figure 1.6B).

15

15

Figure 1.6: Images acquired from phased-array (left) and mechanical rotating element (right)

ICE catheters. Image on the left was acquired within right ventricle (RV) providing a 90-degree

sector view of RV (superior chamber) and LV (inferior chamber). Image on the right features a

mechanical rotating element within a catheter tip (bullseye) in the RA (right atrium) beside

membranous portion of interatrial septum (immediately adjacent to catheter tip between 2 and 7

o’clock). LA, left atrium; IAS, interatrial septum; ICE, intracardiac echo catheter tip.

Thomas Bartel et al. published a review article in the European Heart Journal regarding the

benefits of the use of ICE imaging during TAVI.59 The most pronounced benefit of ICE imaging

compared TEE according to the review was the advantage of ICE being compatible with

monitored anesthesia care without endotracheal intubation (conscious sedation), as mentioned in

the previous section. Further benefits include uninterrupted procedural monitoring, lack of

fluoroscopic interference (which can often be problematic with TEE), and precise Doppler-based

assessment of pulmonary artery pressures.59,60 The article describes how uninterrupted ICE

imaging could be safely performed throughout the procedure with the tip of the catheter

positioned in the right atrium. Live assessment of paravalvular leak post-valve deployment using

ICE imaging is demonstrated in Figure 1.7. ICE is also particularly well suited for monitoring for

catastrophic complications, such as aortic rupture, wire perforation resulting in pericardial

effusion, and coronary occlusion.59 Such complications are almost certainly fatal if not

recognized and acted upon by the operator in a short period of time. Many hybrid catheterization

laboratories can be converted to a traditional cardiac operating room in these circumstances in

order to immediately repair the underlying structure and reverse the effects of the complication.

16

16

Figure 1.7: Use of ICE in monitoring for presence of paravalvular leak (PVL or PL) post-valve

deployment. Position of the ICE catheter within the right atrium in order to image transcatheter

heart valve (THV) in short axis, shown in A. Short-axis images of mild PVL (B), encompassing

< 10% of circumference. Severe PVL demonstrated in C, comprising of > 20% of circumference.

No persistent paravalvular leak demonstrated in D, after post-dilation (balloon expansion) of

valve demonstrated in C. SVC, superior vena cava; RV, right ventricle.59 Reproduced with

permission from Bartel, T., Edris, A., Velik-Salchner, C., & Muller, Silvana. Intracardiac

echocardiography for guidance of transcatheter aortic valve implantation under monitored

sedation: a solution to a dilemma? Eur. Heart. J. Cardiovasc. Imaging. 17, 1-8 (2016).

As the population of patients eligible for TAVI over SAVR continues to grow, and particularly

in the relatively younger demographic (aged 40-70), there will be an increasing imperative for

aggressive valve expansion, and avoidance of catastrophic complications. For example, if a

patient aged 50 undergoes a TAVI, it is expected that with an anticipated valve longevity of 10-

15 years that this patient will likely undergo an additional 2 valve-in-valve replacement

procedures in their lifetime. Each successive replacement adds a new layer of prosthesis at the

level of the annulus (given the old layer is not removed), thus limiting the ability to maximize the

17

17

area of the orifice, or effective orifice area (EOA). Retrospective studies have demonstrated a

clear link between lower starting EOA post-TAVI and clinical events,61 as well as earlier valve

deterioration. In addition, the degree of paravalvular leak is also associated with earlier re-

intervention, which is of particular importance to younger patients given they may have to face

an increase in the number of repeat procedures. ICE is particularly useful in evaluating adequacy

of expansion of the prosthesis post-deployment compared to fluoroscopy, as well as the degree

of paravalvular leak, which will more accurately guide appropriate valve post-dilation. For these

reasons, ICE will potentially impact the durability of the TAVI prosthesis and rates of re-

intervention in these patients in a significant way.

The purpose of our work is to create software for the purpose of tracking the position and

orientation of an ICE imaging surface (either plane or curved surface) within the prior CT

volume using co-registration techniques. By using these software techniques, the program may

act independently of any position sensors or physical tracking devices. We believe that

displaying the ICE imaging surface in context of the three-dimensional surrounding anatomy will

help TAVI operators better appreciate the location of their imaging surface with respect to

neighboring critical structures or physical boundaries and make more informed clinical decisions

while using an ICE imaging catheter.

1.2 Technical Background In our work, we intend to use image-processing techniques to identify the position and

orientation of an ICE imaging surface (either plane or curved surface) in the context of a three-

dimensional CT volume. In broad terms, locating a two-dimensional surface within a three-

dimensional volume may be accomplished either by: 1) hardware methods, such as magnetic

position sensors, or 2) software methods that seek to align common features in the images, such

as image co-registration used in conjunction with a volume search method. The advantages of a

hardware-based solution include: 1) more efficient temporal updating due to the lack of

necessary image processing, and 2) less prone to registration errors caused by the introduction of

new equipment compared to software methods alone.62,63 Hardware methods such as magnetic

tracking systems, however, require dedicated equipment both externally and within the imaging

catheter to function, suffer from concerns regarding accuracy, and are prone to interference from

18

18

external magnetic signals,63,64 which are commonly present in cardiac catheterization

laboratories.

Software methods for determining the position and orientation of a surface within a 3D volume

are able to operate independent of specific hardware, and therefore may be inherently equipment

and equipment vendor-independent. These methods do not interfere with workflow in the

catheterization laboratory or operating room since they do not require any additional external

hardware, such as magnetic position sensors, or shielding from magnetic fields that may interfere

with tracking. Although lag-time from image processing requirements is a key concern regarding

software methods, the use of parallel processing techniques, such as graphic processor units

(GPUs), increases the processing efficiency of these methods by hundreds or even thousands-

fold. The techniques may also be adopted for various combinations of 2D and tomographic

imaging combinations, such as ultrasound-MRI, with only minor adjustments to the underlying

algorithm.

For the above reasons, we have chosen a software-based method for tracking the position and

orientation of a 2D ICE imaging surface within a CT volume. In this section, we will provide a

background on medical image co-registration, which serves as the basis for our method, as well

as prior work in cardiac ultrasound-CT co-registration.

1.2.1 Medical Image Co-Registration

Medical image co-registration is the process of aligning two or more medical images taken with

different acquisition properties, or different forms of contrast in order to benefit from the

information gathered from each imaging modality, and their corresponding overlap. It is

common to perform medical image co-registration for imaging modalities with vastly different

anatomic contrast mechanisms, or to overlap high-fidelity anatomic images with those

demonstrating functional or metabolic information. For example, a clinician may seek to align

magnetic resonance (MR) images of a prostate with transrectal ultrasound (TRUS) in order to

help guide prostate needle biopsy for detection of prostate cancer,65 or combine anatomic images

of a lung tumor derived from CT with images of metabolic activity derived from positron

emission tomography, or PET, in order to better guide radiation therapy.63,66

19

19

Template and Reference Images

Medical image co-registration may be generally viewed as an optimization problem that acts to

minimize an associated cost function as a measure of how closely the two images are aligned.

The problem of medical image co-registration may be broken down into several steps in order to

gain an easier understanding of the overall process. The problem begins with two medical

images; one image serves as the template, T, which will later be transformed, and the other as

the reference (or static) image, R. The purpose of optimization-based co-registration is to

perform some transform of the template image, T, such that it more closely resembles the

reference image, R.

Multilevel Image Representation

Multilevel image representation is a pre-processing step for image co-registration, which

involves developing progressively coarser representations of the medical image data. The

algorithm first attempts to align features in the coarse representation of the image prior to

aligning images containing finer features. Coarse representations of images are generated by

averaging pixel values over a defined space, depending on the ‘level’ of the representation. An

example of multilevel representation involving ICE and CT representations of a silicon cardiac

phantom is shown in Figure 1.8.

20

20

Figure 1.8: An example of multilevel representations: ICE (A) and CT (B) images of a silicon

cardiac phantom (black) in a holder (white) are used for 2D-2D co-registration testing.

Multilevel representations of the overlaid transformed ICE (C-F, red) and CT images (C-F,

background) are shown. The algorithm starts by aligning coarse features (C), and progressively

aligns finer features as more details are added (D-F).

Advantages of performing medical image co-registration in an iterative manner using multilevel

image representations include: 1) optimization is far less computationally expensive in earlier

stages using coarse representations of the images, and 2) by aligning anatomically coarse

features earlier in the process, the algorithm is less likely to get caught within false local minima

when attempting to align finer features. A set of user-defined constraints, or regularizers, act on

the data at each level of image co-registration to prevent the process from tracking within false

local minima. The regularizer effectively prevents large-scale transformations from being

performed between iterations, reducing the scope of the search at each level, and thus successive

21

21

multilevel representations may undergo fewer processing steps at each level, saving a significant

amount of overall computation time.

Distance Measures

The purpose of image co-registration is to minimize the difference between two images, which

can be quantitatively analyzed as a distance measure. In broad terms, a distance measure,

D[T[y], R], is a measure of the similarity between the transformed template image, T[y], and

the reference image, R. It may be measured using a number of different approaches, chosen

depending on the characteristics of the imaging modalities under comparison. Commonly used

distance measures in medical image co-registration include sum of square differences (SSD),

normalized cross-correlation (NCC), mutual information (MI), and normalized gradient fields

(NGF).

In SSD, the ‘distance’ measured between the template and the reference image is based solely on

the sum of the differences between the template and reference squared.67 Mathematically, it is

represented as follows:

𝒟[𝒯[𝑦], ℛ] = 12, -𝒯[𝑦](𝑥) − ℛ(𝑥)23𝑑𝑥

5(Eq1)

W represents the generic interval over which the integration occurs. The SSD distance measure is

most appropriately suited for images with similar contrast, typically acquired with the same

modality, such that a transform of the template produces the reference image.

Normalized cross-correlation assumes a linear relationship between the pixel intensities of the

template and reference images, which can be expressed as: λT[y] = μR, where λ and μ are

scalars. ℝ9 represents the generic interval of real numbers in d dimensions over which

integration occurs.68 An expression to assess the degree to which two images are correlated, <T, R>, is written as:

⟨𝒯, ℛ⟩ = , 𝒯(𝑥)ℛ(𝑥)𝑑𝑥ℝ<

(Eq2)

22

22

In order to normalize this value to a scalar between 0 and 1, a normalized cross-correlation

divides the above value by the product of the square root of the template and reference images

correlated with themselves, such that:

NCC[𝒯,ℛ] = ⟨𝒯, ℛ⟩‖𝒯‖‖ℛ‖ ,where

‖𝒯‖ = D⟨𝒯, 𝒯⟩(Eq3)

The above measure of NCC produces a value that reaches a maximum of 1 when the images are

the same within a scalar term. In order to use NCC in a traditional minimization problem, the

distance measure is expressed as:

𝒟HII[𝒯,ℛ] = 1 − NCC[𝒯,ℛ]3 = 1 −⟨𝒯, ℛ⟩3

‖𝒯‖3‖ℛ‖3 (Eq4)

The values of the NCC in the equation are squared in order to place equal weighting for directly

proportional, and inversely proportional correlations, thus imaging modalities with inverted color

scale are also viewed as highly correlated. An example of this principle is brain MR images

acquired using T1 and T2-weighted protocols.

Mutual information assumes no linear relationship between the information contained in T and

R. A histogram of the values contained in T and R is created, which summarizes the

coincidences of all possible pairs of values that can be compared between them.69,70 A measure

of the entropy, H, reflects the ‘sharpness’ of the histogram, and is a key determinant of the

distance measure. Due to the complexity of the mathematical underpinnings of this distance

measure, we will refer the reader to Chapter 7 in ‘FAIR: Flexible Algorithms for Image

Registration’ by Jan Modersitzki for further details.71 Mutual information is particularly well

suited for imaging modalities with drastically different contrast mechanisms, such as comparing

tomographic imaging techniques (CT, MR) with metabolic imaging (PET, SPECT, Thallium).70

The normalized gradient fields (NGF) distance measure is based on comparing the normalized

pattern of image intensity changes between the template and the reference. It is far less restrictive

than the SSD and NCC methods, which rely heavily on comparing the pixel values throughout

the image, however far less general than the MI method, which analyzes the entropy of a mutual

23

23

information histogram. The concept of using changes in image intensity to compare two imaging

modalities was borne out of the observations that: 1) intensity changes naturally play a dominant

role in co-registration processes, and 2) anatomic boundaries with sharp changes in image

intensity are often well-preserved between imaging modalities. In other words, areas of image

intensity changes in one imaging modality should correspond to the image intensity changes

based on anatomic boundaries in another.

To perform analysis based on NGF, gradient representations of the images are created in order to

map the changes in image intensities.72,73 Given the magnitude of change varies across different

imaging modalities, gradient representations, such as ∇T, are divided by the magnitude of the

entire gradient field, |∇T|, thus focusing solely on locations of changes within the image rather

than their magnitude. The normalized gradient field of the template image, n[T] or n[T, η], is

defined as:

𝑛[𝒯] = 𝑛[𝒯, 𝜂] = ∇𝒯

D|∇𝒯|3 +𝜂3(Eq5)

where η is a scalar which serves as an ‘edge parameter’, such that |∇T| > η at a point that is

considered an edge, and |∇T| < η at a point which is considered within the noise level. For

example, by reducing η, the distance measure is more sensitive to detect edges, however will be

more susceptible to falsely assuming that noise-level variations represent anatomic boundaries.71

A comparison of the gradient representations of the template, n[T], and the reference, n[R], images is performed by determining the area spanned by the two vectors as a measure of linear

dependency. This is performed by multiplying the transpose of the template by the reference, and

subsequently squaring the result in order to remove bias for the absolute direction of change. The

resulting value, (n[T]T . n[R])2, is a maximum when edges align, and thus to use this in a

traditional minimization problem (over interval W), it is re-termed as: 1- (n[T]T . n[R])2. Thus,

mathematically, the minimization problem for NGF may be expressed as:

𝒟HQR[𝒯, ℛ] = NGF[𝒯,ℛ] = , 1 − (𝑛[𝒯(𝑥)]U5

𝑛[ℛ(𝑥)])3(Eq6)

24

24

Regularization and the Ill-Posed Problem

Co-registration may be viewed as an ill-posed problem resulting in a highly non-convex cost-

function (Figure 1.9).74,75 In a non-convex circumstance, several unique and independent

solutions can be inferred at locations of various independent local minima. A way to resolve the

discrepancy of multiple solutions to an ill-defined problem is to introduce regularization, a term

whose purpose is to modify the problem such that there is a unique solution, or resolve the non-

convexity of the cost function. If multiple potential solutions exist, a term that differentiates the

quality of the solutions can lead to one that is unique based on user-defined constraints.

Figure 1.9: Graphical examples of convex (left) and non-convex (right) cost functions.74

Reproduced with permission from Reali, F., Priami, C., & Marchetti, L. Optimization algorithms

for computational systems biology. Front. Appl. Math. Stat. 3, 6 (2017).

Elastic-based regularization is the most common form of regularization used in contemporary

image co-registration algorithms to solve the issue of non-convexity of the cost function. An

elastic regularizer is based on the concept of potential energy created by deforming an elastic

material.76 It is applied to a non-convex cost function as a method to penalize solutions requiring

large transformations (in an elastic manner) in order to optimize the distance measured.

Mathematically, it is expressed as:

𝒮[𝑦] = 12,

|ℬ[𝑦]|3𝑑𝑥,whereℬ = YD𝜇∇ 00 D𝜇∇

Dλ + 𝜇𝜕^ Dλ + 𝜇𝜕3

_5

(Eq7)

25

25

λ and μ represent the Lamé constants, which are material-dependent quantities that arise in

elastic stress-strain relationships. In the elastic stress-strain model, μ can be thought of as the

shear modulus. Ñ represents the divergence operator, and 𝝏1 and 𝝏2 represent first- and second-

order differential operators. W represents the generic interval over which the integration occurs.67

As mentioned previously, the main advantages of using regularizers in the image co-registration

process are to: 1) improve image co-registration accuracy by preventing the process from

tracking within false local minima, and 2) improve processing efficiency by limiting the scope of

transformations, and thus processing steps or computation time, at each iteration of a multilevel

image representation.

Transformations

In the co-registration process, iterative transformations of the template image, T, are performed,

and the resulting image is compared to the reference image, R, at each iteration by means of the

distance measure and the regularizer. If a transformation results in a smaller ‘distance’ measured,

the resulting template image may be viewed as more ‘similar’ to the reference. As mentioned

previously, transformations of the template image, T[y], may be classified as rigid

transformations, involving only rotation or translation, or spline-based transformations, which

also includes more free-form transformations such as shearing and scaling of individual

components of the image.77

Mathematically, a 2D rigid transformation is expressed as:

𝑦^ = cos(𝑤^) 𝑥^ − sin(𝑤^)𝑥3 +𝑤3

𝑦3 = sin(𝑤^) 𝑥^ + cos(𝑤^)𝑥3 +𝑤h

ory=wx,wherew=[𝑤^;𝑤3;𝑤h] ∈ ℝh(Eq8)

For each point (x1, x2) in an image, a rigid translation can be defined into another space (y1, y2).

w = [𝑤^; 𝑤3;𝑤h] denotes a translation vector. ℝh denotes real numbers in three dimensions.

Rigid transformations are particularly useful in maintaining computational efficiency in the co-

registration process given they are limited to three degrees of freedom (DoF). The computational

26

26

efficiency is achieved at a cost of robustness, especially in comparing imaging modalities with

vastly different contrast mechanisms and artifacts, which may somewhat distort geometry in

different manners.

Spline-based transformations require a greater deal of computational processing given the ability

to act differently on individual components of the image. Mathematically, they are far more

complex than rigid body transformations:

𝑦^ = 𝑥^ + n n 𝑤op,oq^ 𝑏op(𝑥^)𝑏oq(𝑥3

sq

oqt^

)andsp

opt^

𝑦3 = 𝑥3 + n n 𝑤op,oq3 𝑏op(𝑥^)𝑏oq(𝑥3

sq

oqt^

)(Eq9)sp

opt^

In the equations above, bj denotes the splines, w1 and w2 denote the coefficients of y1 and y2

respectively, and p1 and p2 are the number of coefficients in the spline expansion.

In order to limit the computational processing requirement of spline-based transformation,

regularizers places constraints on which types of transformations can be reasonably performed

for the data set. For example, elastic regularizers will limit transformations that result in

significant asymmetric compression, or scaling, of individual image components due to the large

‘elastic potential energy’ that would be required to perform such a transformation.

Spline-based transformations are particularly useful in circumstances where anatomic boundaries

may be ‘warped’ compared to one another due to differences in image acquisition properties. It is

important to reconcile, however, whether anatomic boundaries align due to the deformation

process itself, rather than common anatomic boundaries.

Putting Together the Optimization Problem

Co-registration may be viewed as an optimization problem whose aim is to find a transformed

version of the template image, T, which most closely resembles the reference image, R. The

optimization framework is expressed mathematically as follows:

27

27

𝒥[𝑦] = 𝒟[𝒯[𝑦], ℛ] + 𝑆[𝑦]z{⎯} min (Eq 10)

In the above formula, J[y] is the parameter being optimized, known as the joint functional, y

represents the transformation, D[T[y], R] is the distance measured between the transformed

version of the template, T[y] and R, and S[y] is the regularizer (or set of constraints) applied to

the transformation, y.

In the optimization process, the template and the reference images first undergo pre-processing

steps, where multilevel representations of both images are created. The multilevel images are

generated using a defined quantity of pixel averaging, which is used to define each ‘level’ of

image representation. At the lowest defined level of image representation, the template image, T,

enters the processing loop, and is initially compared to the reference image, R, using the pre-

defined distance measure and the regularizer term. The regularizer term differentiates the quality

of the solution based on user-defined constraints. In the case of elastic regularization, it is the

elastic potential energy difference between two versions of the image based on the

transformation required to produce them. The joint functional, J, is used to determine the

‘difference’ between the template image, T, and the reference image, R, using the combination

of the distance measure, and the additional values introduced by the regularizer term. An optimal

transformation is determined based on the transformation producing the lowest value in the cost

function (i.e. lowest J-value). As the loop repeats, the difference in the joint functional, J,

between the images in the previous and the current iterations is used to determine how much

‘optimization’ has occurred in the last round, denoted by J-Jold. During iterations of the

optimization process, the value of J-Jold continues to decline when the template appears more

similar to the reference in successive steps. A stopping condition is met when J-Jold crosses a

pre-determined lower threshold value. When the stopping condition is met, the transformed

template is thought to be as close to the reference as reasonably achievable. A graphical

representation of the algorithm at each multilevel representation is illustrated in Figure 1.10.

The algorithm repeats this optimization process for each level in the multilevel representation,

typically three to four times, based on user specification. Given the initial iteration is performed

using a coarse representation of the image, coarse image features are aligned first in the co-

registration process. Successive iterations start with a version of the template image transformed

28

28

from the previous iteration. The use of regularizers in successive steps ensure that alignment of

finer features does not deviate significantly from alignment of coarse features in prior iterations.

The 2D-2D ICE-CT co-registration process in our work performs the co-registration process

outlined above with three multilevel representations, normalized gradient fields as the distance

measure, elastic regularizers, and spline-based transformations.

Figure 1.10: 2D-2D Co-Registration Loop: 2D ICE surface (template) and 2D CT slice

(reference) undergo initial pre-processing (multilevel representation). Template and reference

images are compared using a chosen distance metric (NGF) to produce a distance measure. A

regularization term (elastic regularization in our case, S) sets the search constraints (ie. penalties

applied for large transformations). The template is subsequently transformed using a pre-

determined series of transformation (spline-based transformations). A joint functional, J, which

is a linear combination of the distance measure and the regularizer term is determined as a as a

measure of the quality of the overlap between the template and the reference. The process is

repeated until a pre-determined stopping condition (ie. threshold J-value) is met.

29

29

1.2.2 Search Method within the CT Volume

Geometric Representation of the ICE and CT Image Surfaces

In our work, the volume search method consists of performing an exhaustive search of the

position and orientation of a 2D ICE imaging surface within a 3D CT volume. Due to acquisition

properties of the ICE imaging catheter, the 2D ICE imaging surface is represented as a shallow

cone in three-dimensional space specified by the ICE catheter (usually with 10 degrees

curvature). At the centre of the ICE image is the ICE catheter itself, which is subtracted from the

image during analysis, and thus the ICE imaging surface is represented in 3D space by a shallow

cone with a missing section at the apex that is the radius of the ICE catheter. Subsequent

comparisons to CT data in our work require comparing the conical ICE imaging surface to CT

data with similar conical geometry extracted from the 3D CT volume. Shallow conical

geometries are extracted from the 3D CT volume by extracting CT data points according to the

coordinates of the surface of a conical mesh. The conical representations of both the ICE and CT

data are compared for the purpose of the co-registration algorithm.

Constructing the Cost Function

The joint functional, or J-value, of each individual 2D-2D image co-registration represents a

measure of the quality of the overlap. If multiple co-registrations are performed using CT

surfaces from various positions and orientations within the CT volume and a single ICE image,

the co-registration with the lowest J would represent the likeliest position and orientation of the

ICE image within the CT volume. For the purpose of our work, a 5-dimensional matrix of J-

values resulting from co-registration of CT surfaces at defined x, y, z positions within the CT

volume and at defined pitch and yaw angles (in-plane rotation addressed by 2D-2D co-

registration process) represents the cost function of the search. In our work, the cost function

represents a search within a given space (for example a 32 mm width and 16 degree arc) with a

defined resolution (for example 1 mm spacing and 1 degree resolution). The resulting cost

function in our example containing 32 points in each x, y, z direction, and 16 points in each pitch

and yaw rotation contains over 8 million individual elements, and thus processing shortcuts to

improve efficiency of the search are necessary to significantly reduce processing time.

30

30

Optimizing Efficiency of Volume Search

The shape of the associated cost function will heavily influence the strategy employed for

searching for a 2D surface within a 3D volume. In a circumstance of a highly convex cost

function (Fig 1.9a), a gradient search algorithm such as the gradient descent method is a

computationally efficient way to solve the optimization problem. In a gradient descent method, a

limited initial sampling of the cost function is performed. The derivative of the cost function of

the initial sampled version will indicate the areas and direction of the steepest negative slope,

which will be used as a starting point for a more detailed (or well-sampled) search. The process

continues in the direction of the slope until a unique solution is found at the minimum of the

curve. This method is used successfully in examples where solutions converge on a unique

global minimum.

In a problem involving a highly non-convex cost function (Figure 1.9B), such as the co-

registration methods used in our work, a more exhaustive form of searching within a 3D volume

is performed. To maintain computational efficiency, our method uses a series of under-sampled

passes in order to perform more detailed sampling only within the troughs of the cost function.

Each pass (n=4, 3, 2, 1) under-samples the data at a ratio of 2n-1 (8:1, 4:1, 2:1, 1:1), however only

the J-values falling below (100% - (n x 10%)) of the range of J-values (|Jmax – Jmin|) will be

sampled in the next layer. For example, the first layer is under-sampled at a ratio of 8:1, and only

the data falling below 60% of (|Jmax – Jmin|) will be sampled in the next layer. The second layer is

under-sampled at a ratio of 4:1, and only the data falling below 70% of (|Jmax – Jmin|) will be

sampled in the next layer, and so on. The above formulae were derived empirically from

experimentation with a model dataset.

A final step to increase processing efficiency is the high-resolution position and angle update

performed at the end of the volume search. For example, if the lowest position resolution using

the above sampling technique is 2x, the data surrounding the solution may be sampled with a

resolution of 1x by fixing the angle and searching locally for the best position. The algorithm

subsequently fixes position and searches locally for an angle with 1x resolution. This step may

be included if the resolution of the CT volume is less than the resolution of the volume search.

The result is an efficient search algorithm that performs high-resolution sampling only at the

lowest points of each trough where the optimization solutions are found. In an example of a

31

31

search using our algorithm with a 32 mm and 16-degree radius and 5 degrees of freedom, a 1:1

exhaustive search would require processing 8,388,608 co-registrations, however using the

progressive sampling method listed above, it is reduced to 174,618 co-registrations (48 times

faster) without suffering a theoretical loss of accuracy.

1.2.3 Prior Work in Cardiac Ultrasound-CT/MR Co-Registration for Use in Procedural Navigation and as an Adjunct to Magnetic Tracking Systems

Prior research in the field of intra-operative or intra-procedural co-registration of cardiac

ultrasound with a pre-procedural tomographic imaging modality, such as CT or MR, has been

conducted for various applications. Significant contributions to the field have been made by Dr.

Terry Peters and his team at the Robarts Research Institute in London, Canada.64,78–86 Co-

registration and hardware tracking efforts from this group using cardiac ultrasound and/or

tomographic imaging include: 1) magnetic position sensing of instruments during cardiovascular

interventions,62,63 2) co-registration of CT with TEE,78–83,85,86 3) use of pre-operative 4D

tomographic images,86,87 and 4) embedding a phased-array ICE catheter within the delivery

catheter combined with magnetic tracking.85 A number of the above projects were performed

with the main purpose of reducing contrast exposure during minimally invasive cardiovascular

procedures, which may reduce the risk of long-term kidney injury caused by iodine-based

contrast agents.

An example of magnetic position sensing alone includes a system for magnetic tracking during

deployment of thoracic aortic stent grafts.62 In this example, magnetic tracking sensors were

embedded in the device catheter and cannula for a thoracic aortic stent, as well as the ultrasound

probe to gauge the position of the device with respect to the intra-operative live ultrasound image

plane. Using the live ultrasound and magnetic position tracking information, it is possible to

perform this procedure in an x-ray fluoroscopy suite with minimal or no exposure to iodine

contrast. Tracking in this application was performed with localization errors of 1.4 mm and 4.2

mm in phantom and animal experiments respectively.

Examples of co-registration of ultrasound with CT also include combined systems involving the

use of both magnetic tracking and software image co-registration. In one example, the position of

32

32

the TEE probe contained a magnetic tracking system whose tracked position and orientation

would be used to determine the corresponding 2D plane in the pre-operative 3D CT volume.79

Retrospectively gated CT acquisitions were obtained at ten different points in the cardiac cycle in

the pre-operative setting, thus creating a 4D CT data set. Image co-registration between the live

ultrasound image and the CT plane at the corresponding point in the cardiac cycle (based on

ECG gating) was performed with a target registration error of 1.7 mm. Another example

incorporated an ICE imaging catheter within the delivery system with a magnetic tracking

system used to determine its position and orientation.85 Live ICE-CT co-registration could have

been performed using this technique, similar to the prior example; however, this was not

included in the paper.

The above examples include the use hardware tracking systems due to the computational

complexity involved in software-only methods. With further advances in the field of parallel

processing techniques, predominantly the use of GPU-based hardware acceleration, software-

only methods are potentially realizable with almost real-time temporal resolution.

33

33

Chapter 2

ICE-CT Co-Registration for the Purpose of Guiding Cardiovascular Interventions

2.1 Introduction Imaging plays a crucial role in performing transaortic valve implantation (TAVI), including the

use of tomographic imaging (such as CT or MR) for procedural planning and x-ray and

ultrasound-based techniques for live imaging guidance.6–8,14,15 In the earlier experience with

TAVI, procedures were almost exclusively performed under general anaesthesia with the use of

transesophageal echocardiography (TEE) and fluoroscopy, which provided the operator with

soft-tissue contrast and Doppler flow assessment in real-time.11–13 Advances in the use of CT for

the purpose of pre-procedure measurements and determining optimal intra-procedural

fluoroscopic angles have enabled operators to perform TAVI successfully using only

fluoroscopy.14,15,55 By relying solely on fluoroscopy, the majority of TAVI operators have

transitioned from performing procedures under general anaesthesia to performing them under

heavy conscious sedation without the need for intubation.17 Although reducing the rates of

general anaesthesia has effectively improved procedural safety and post-operative inpatient

length of stay, the loss of soft tissue contrast and hemodynamic information afforded by TEE has

some considerable drawbacks, including limitations in quantifying the degree of paravalvular

leak (PVL) post-valve deployment, which is well-correlated with adverse clinical events.53

Intracardiac echocardiography (ICE) represents a novel alternative to TEE for intra-procedural

imaging guidance for TAVI.59 ICE is an ultrasound-based imaging technology which uses either

a single rotating mechanical element or phased-array ultrasound transducer at the distal end of

the catheter to provide images with soft tissue contrast and colour Doppler information in real-

time without the need for increased levels of sedation. A common concern among early-

experience operators with ICE is the difficulty in appreciating the three-dimensional context of

the acquired ultrasound images. Given patients undergoing TAVI require a planning CT study, a

solution demonstrating the position and orientation of the ICE imaging surface in context of the

previously acquired 3D CT anatomy would be highly desirable.

34

34

Methods for determining the position and orientation of an ICE imaging surface within a 3D

image volume interprocedurally, including hardware and software techniques have been

previously attempted. Please refer to section 1.2.3 for further details regarding prior navigation

attempts using hardware techniques.

The use of software tracking methods based on alignment of anatomic features within the images

represents a robust approach in determining position and orientation of the catheter tip, however

at a cost of significant computational complexity. The main advantages of using such techniques

are the lack of dedicated hardware, interference with operative workflow, and interference from

external electromagnetic signals. Medical image co-registration may be used in these methods to

test the alignment of anatomic features within the images and provide metrics of the quality of

the alignment produced. The significant challenge of aligning images from two drastically

different modalities may be overcome by extracting well-preserved features such as major

anatomic boundaries in the comparison. Combined with three-dimensional search techniques,

these methods may be used to determine which image pairs produce the optimal alignment, and

thus reflect the most likely position and orientation of the catheter tip within the 3D volume.

Three-dimensional searches are limited in scope by only searching a small volume around the

last known location. Based on these principles, we present a system for determining the position

and orientation of an ICE imaging surface within a 3D CT volume using automated co-

registration methods for the purpose of improving imaging guidance using ICE during TAVI.

2.2 Methods

2.2.1 Software environment Software for the purposes of our study was developed in the MATLAB programming

environment (MATLAB & Simulink, Mathworks Inc, Natick, MA), using the Flexible

Algorithms for Image Registration (FAIR) package (Jan Modersitzki, et al, Institute of

Mathematics and Image Computing, University of Lübeck, Germany). A custom user interface

was developed for this application that demonstrates the current ICE imaging surface, the 3D CT

volume, and a three-dimensional representation of the position and orientation of the ICE surface

(shown in Figure 2.1). Using the software interface, the user is able to select the initial position

and orientation of the search, define the search constraints (including position and angle

35

35

constraints), the degree of curvature of the ICE imaging surface (as specified by the ICE

catheter), and sampling interval of the initial search. The conical representation of the ICE

imaging surface in 3D space may either be represented as the ICE image, or as a CT

representation of the ultrasound surface (as seen in Figure 2.1). Extracting CT data along this

geometry was created using the Cone MATLAB plugin (Waldemar Swiercz, et al, Harvard

University, MA) to define the inner and outer cone radii. Polar images were constructed by

extracting data along the lines connecting points at defined angles along the inner and outer radii.

Polar images were subsequently converted to Cartesian images, thus resulting in planar

representations of the conical geometry. Representing the CT data in this format allowed for

comparisons of the conical ultrasound geometry to similar geometry extracted from the 3D CT

volume.

Figure 2.1: Single 2D ICE imaging surface (top, left) and 3D CT volume (bottom, left) are

loaded into custom user interface. CT representation of the ICE imaging surface (right, blue) in

best estimated position and orientation is shown in context of CT volume.

2.2.2 Process Overview The search for the estimated position and orientation of the ICE imaging surface within a 3D CT

volume begins with a user-defined best estimate. The user starts by selecting their estimate of the

initial position and orientation of the ICE catheter tip within the CT volume. A location with

36

36

clearly defined unique anatomic features is advised as a starting point to reduce error introduced

in this first step. Following the initial ‘user defined’ step, the algorithm performs an automatic

local search in order to optimize position and orientation, and subsequent steps of the algorithm

are performed automatically within a limited search space. The user may specify the maximum

distance and orientation changes involved in the search, as well as the resolution (ie. spacing

between points) in the search. For example, for the purpose of our experiment, the maximum

distance searched for each step was 16 mm from the starting point (total 32 mm width) with a 0.5

mm final resolution, and the orientation search was limited to a 16-degree arc (-8 degrees and + 8

degrees from the starting orientation) with a 1-degree final resolution. The approach of using

limited local searches between each step was chosen due to processing efficiency, without

suffering a significant loss in accuracy as long as the catheter tip did not move outside of the

search space between steps. A local search strategy represents a reasonable method for tracking

the location of an ICE catheter since it typically does not undergo large rotations and translations

within a limited time period during invasive cardiac procedures.

The search for position and orientation of the ICE catheter within a limited volume at each step

comprises of numerous individual 2D-2D image co-registrations at various positions and

orientations. The algorithm extracts a planar image representation of a conical CT surface at a

specified position and orientation, and with a pre-defined curvature angle (based on the curvature

of the ICE surface). A 2D-2D image co-registration is performed between the extracted CT data

and the ICE image (outlined in detail in section 2.2.3). A resulting value, J, known as a joint

functional, is derived from the 2D-2D co-registration algorithm and quantifies the quality of the

overlap between the two images. The algorithm repeats this process for each position and

orientation within the local search volume at a pre-determined spacing interval. The matrix of J-

values at various positions and orientations thus represents a multidimensional cost function. The

position and orientation with the lowest J-value, or lowest point in the cost function (ie. the best

overlap of the images), represents the estimated position and orientation of the ICE catheter. An

exhaustive search method is used for the position and orientation search due to the presence of

multiple minima and non-convexity of the cost function. A strategy of under-sampling the cost

function, and subsequently performing more detailed sampling of data falling under a certain

pre-defined threshold was performed (outlined in detail in section 2.2.4). The algorithm thus

employs an exhaustive search of a cost function derived from joint functional values from

37

37

individual 2D-2D co-registration between a static ICE image and CT images at various positions

and orientations to determine its estimate of the position and orientation of the ICE catheter tip.

2.2.3 2D-2D Image Co-Registration Image Pre-Processing

For the purpose of the co-registration process, the CT data will be identified as the reference

image, and the ultrasound image as the template image. ICE and CT images underwent initial

pre-processing where fine features were smoothed in a stepwise manner, creating what is known

as a multilevel representation. The use of multilevel representation in our work is explained

further in section 1.2.1 (under Multilevel Image Representation heading). Coarse representations

of the images were aligned using the 2D-2D co-registration algorithm, the output of which

served as the starting point for the next iteration of the algorithm. Subsequently, less smoothed

versions of the images underwent the 2D-2D co-registration process. The images underwent this

process three times prior to final image alignment.

Distance Measure

Normalized gradient fields (NGF) were used as the distance measure for the 2D-2D co-

registration process. The distance measure, D[T[y], R], is a measure of the similarity between

the transformed template image, T[y], and the reference image, R. Further details regarding the

use of NGF as a distance measure in the 2D-2D co-registration process may be found in section

1.2.1 (under Distance Measures heading). Gradient representations of the ultrasound and CT

image were used in our work since they represent the location of image intensity changes, rather

than reflecting the absolute magnitude of the intensity changes. The use of NGF as a distance

measure therefore is ideally suited for comparing vastly different imaging modalities, which both

contain well-preserved anatomic boundary information, however with differing magnitudes of

change at boundaries, such as those seen in ultrasound and CT.

Regularization

Elastic-based regularization was employed in our study to help resolve the non-convexity of the

cost function. An elastic regularizer is based on the concept of potential energy created by

deforming an elastic material. When used in the context of comparing two images, quantitative

38

38

penalties are applied for large transformations in an elastic manner when compared to the

previous iteration. Further details may be found in section 1.2.1 (under Regularization and the

Ill-Posed Problem heading).

Transformation

Spline-based transformations were used for the purpose of our study, which includes rotation,

translation, and some free-form transformations such as rotation, shearing and scaling of

individual components of the image. Non-rigid transformations are useful in comparing images

from imaging modalities with vastly different acquisition mechanisms given the distortions and

artifacts that may occur at anatomic boundaries. The use of regularization limits substantial

image deformation that would potentially cause a reference image to co-register to a significantly

distorted version of the template image, rather than the image itself. Further details may be

found in section 1.2.1 (under Transformations heading).

Solving the 2D-2D Optimization Problem

The optimization framework used in our work to find a transformed version of the template

image, T, which most closely resembles the reference image, R, is expressed mathematically as

follows:

𝒥[𝑦] = 𝒟[𝒯[𝑦], ℛ] + 𝑆[𝑦]z{⎯} min (Eq 10)

In the above formula, J[y] is the parameter being optimized, known as the joint functional, y

represents the transformation, D[T[y], R] is the distance measured between the transformed

version of the template, T[y] and R, and S[y] is the regularizer applied to the transformation, y.

In our work, the template and reference images are pre-processed as multilevel representations

prior to entering the co-registration loop. The template and reference images are compared using

the NGF distance measure and regularization terms. A pre-prescribed transformation is

performed and the loop repeats with the transformed template image compared to the reference

image. If the J-value between the images is less than the prior iteration, the transformed image

serves as the starting point for the next iteration. The loop repeats until the difference between

the J-value of the prior iteration and the current iteration reaches a lower threshold value. The

process repeats a total of three times, one for each set of multilevel representations. The resulting

39

39

transformation of the prior multilevel representation serves as a starting point for the next

iteration. Further details regarding this process may be found in Section 1.2.1 (under the heading

Putting Together the Optimization Problem).

2.2.4 Search Method within the CT volume Cost Function

For the purpose of our work, the cost function for the localization search was constructed using

joint functional values, or J[y], at each of the x/y/z translation and pitch/yaw rotation steps in the

volume search. J[y] may therefore be viewed as a cost function in five dimensions. Further

details regarding constructing the cost function may be found in Section 1.2.2 (under the heading

ConstructingtheCostFunction).

Optimizing Search Efficiency

Efficient and accurate processing of a cost-function representing a 5-DoF search within three-

dimensional space requires several designed shortcuts in order to reduce computation time. For

the purpose of our work, the 5-DoF search began with an initial user-defined starting point as a

‘best estimate’ of the position and orientation of the ICE imaging surface within the CT volume.

In each iteration of the 5-DoF search, the search was limited within a 16 mm translation distance

from the starting point (total 32 mm search width) and 16 degrees of rotation (-8 to +8 degrees

pitch and yaw). Given an ICE catheter does not typically perform large rotations and translations

suddenly during an invasive cardiac procedure, the limited search scope represents a reasonable

shortcut.

The matrix of J-values derived from the 5-DoF search within three-dimensional space is an

inherently non-convex cost-function with several local minima. An exhaustive search strategy

must therefore be employed to better resolve the difference between these local minima. A

layered progressive under-sampling strategy was used in our work in order to maintain

processing efficiency. AfinalsteptoincreaseprocessingefficiencyistheHigh-Resolution

PositionandAngleUpdateperformedattheendofthevolumesearch.Further details

regarding the Progressive Sampling Method and High-Resolution Position and Angle Update

40

40

used in our work may be found in section 1.2.2 (under the heading Optimizing Efficiency of

Volume Search).

When an updated ICE image was received, the algorithm would repeat the above process using

the position and orientation determined by the previous iteration as the starting point. The GUI in

MATLAB displayed the updated plane position and orientation in 3D space in the context of

axial and sagittal CT planes, as shown in Figure 2.1.

2.3 Experimental Setup A custom silicon cardiac phantom was used for the purpose of our study. CT images were

acquired using a Toshiba Acquilion One™ CT scanner (Toshiba Inc, Tokyo, Japan). ICE images

of the phantom were acquired using a Conavi Foresight™ ICE imaging catheter (Conavi Medical

Inc, Toronto, Canada). The silicon phantom was placed in a custom water bath container, which

contained a hemostatic valve built into its side to allow passage of the ICE catheter without

leaking the water content. The ICE catheter was placed in an initial position, and both ICE

images and CT volumes were acquired at this location, and 9 additional manual pullback

positions. The pullback positions were each not a standardized length, however did not exceed

the maximum distance (16 mm) searched by the algorithm during each iteration. CT scans were

performed with the ICE catheters at each pullback location. The CT volumes were used to

confirm the position and orientation of the ICE catheter tip at each pullback location. A final CT

volume was acquired with the ICE catheter removed from the phantom as the CT data set for the

automated localization portion of our study.

CT images were exported in DICOM format, including standard DICOM headers. ICE images

were exported to PGM format via third-party image-processing software from Conavi Medical

Inc. Header information including appropriate scaling and the conical angle of the ICE surface

were exported from the third-party software. Image data were extracted from the respective

DICOM and PGM formats, and corrective scaling of the images were applied in MATLAB.

Initially, the first of ten ICE images and the CT volume without the ICE catheter were loaded in

our custom software. The experiment began with a manual user-defined best estimate of the ICE

41

41

catheter position and orientation. Following this first step, the algorithm updated the initial

position based on the CT volume search described in section 2.2. The true position and

orientation of the ICE imaging catheter tip was known based on the CT scan acquired with the

ICE catheter in its initial position. Localization error for position was defined as the three-

dimensional distance between the true tip position and the algorithms’ estimated position.

Localization error for orientation was defined as the three-dimensional difference in pitch and

yaw between the known and estimated orientations. In-plane rotation, or roll, was not factored

into the error calculation.

The final estimated catheter position for each pullback step served as the starting point for the

next search. A total of ten catheter tip position and orientation estimates were performed in

succession by the algorithm. It is possible that the position of the next pullback location falls

outside the search radius if the error introduced by the preceding step in addition to the distance

in the current step is larger than the search radius. Using our software, it is possible for the user

to perform a manual correction if the estimated position strays too far from the true catheter tip

location. We did not employ this strategy for the purpose of our experiment.

2.4 Results A detailed description of the steps performed in the algorithm are outlined in section 2.2. In

brief, the algorithm performs a series of 2D-2D co-registration steps with a static ICE image and

conical CT surfaces at various positions and orientations within the CT volume. Each 2D-2D co-

registration produces a joint functional, or J-value, which is used to construct a 5-dimensional

cost function. Given the J-value represents the quality of overlap of the images in the 2D-2D co-

registration, the position and orientation of the lowest J-value represents the most likely position

and orientation of the ICE image within the CT volume. We present here the results of the 2D-

2D co-registration process and the search method within the CT volume, which comprises of

numerous 2D-2D co-registrations.

42

42

2.4.1 2D-2D Co-Registration 2D-2D ICE-CT co-registrations were performed in our work using: 1) multilevel representation,

2) normalized gradient fields (NGF) distance metric, 3) elastic regularizers, and 4) spline-based

transformation. The results of a sample 2D-2D co-registration are shown in Figure 2.2. In this

illustration, the red outline in Figure 2.2 C-E represents the contour of the simulated RV contour

(7 o’clock to 2 o’clock in Figure 2.2 B) during the co-registration process in context of the CT

data. Figure 2.2 C-E demonstrate multilevel image representations of the ICE (red) and CT data

(background). Initial registration is performed by aligning coarse features of the transformed ICE

image and CT (C). Finer features are aligned progressively as further details are added to the

images (D,E).

Figure 2.2: An example of 2D-2D Co-Registration: ICE (A) and CT (B) images of a silicon

cardiac phantom (black) in a holder (white) are used for 2D-2D co-registration testing.

Multilevel representations of a portion of the transformed ICE image (C-F, red) and the entire

43

43

CT image (C-E, background) are shown. The algorithm starts by aligning coarse features (C),

and progressively aligns finer features as more details are added (D,E).

2.4.2 Search Method within CT Volume The experimental setup for our work is outlined in Section 2.3. Localization error for position

was defined as the three-dimensional distance between the true tip position determined by CT

and the position estimated by the algorithm (Figure 2.4). Localization error for orientation was

defined as the three-dimensional difference in pitch and yaw between the known and estimated

orientations. In-plane rotation, or roll, was not factored into the error calculation. The average

localization error for position and orientation over the ten pullback positions used in our

experiment were 4.1 mm and 8.1 degrees respectively. Figure 2.3 illustrates the estimated

position and orientation of the ICE imaging catheter traveling through a cardiac phantom from

the apex to the base of the right side of the heart (top row) compared to the actual trajectory

verified by CT scans (bottom row). The relatively high average error for the orientation estimates

was heavily influenced by the rotational symmetry of the cardiac phantom about the x-axis (or

pitch; refer to Discussion section for details).

Figure 2.3: Example of the algorithm following a live ICE imaging plane through 3D CT

volume. Top row represents the estimated trajectory of the ICE catheter based on the volume

search method. Bottom row represents the actual trajectory of the catheter based on identified

44

44

position and orientation on CT scans. In both rows, the CT representation of initial position of

the 2D ICE image in the silicone phantom shown on right. Ten subsequent CT representations of

ICE imaging surface position and trajectory (red line) shown in middle. Final position and

trajectory are shown on right side.

Localization error as described above was calculated for each of the ten steps in the experiment.

Each step may be viewed independently as a process that begins at the ‘estimated prior location’

and attempts to locate the ‘actual current location’ (Figure 2.4). Thus, the ‘distance between the

prior estimated location and current actual location’ must fall within the ‘maximum search

distance’ (in our case 16 mm and 8 degrees) in order for the algorithm to be able to identify the

correct current position and orientation.

Figure 2.4: Graphical representation of locations and distances during each pullback step.

Estimated prior location serves as a starting point for the search. The distance between the prior

estimated location and the actual current location represents the distance from the prior step to

the current true position. The distance between prior and current actual locations is the distance

travelled by the catheter during each pullback. Localization error is defined as the distance

between the estimated and actual locations. The maximum search distance represents the

maximum distance the algorithm searches from its starting point. The total distance searched in

each dimension is twice this value.

Figure 2.5 and 2.6 demonstrate the localization error for position and orientation as a function of

the pullback step number in millimeters and degrees respectively (blue lines). The distances

between the starting position and orientation for each pullback step and the actual position and

45

45

orientation for each pullback in millimeters and degrees respectively are shown as green lines.

The distance between actual location of prior iteration and actual location of current pullback

step in millimeters is shown as a purple line in Figure 2.5. Given the angle was fixed throughout

the experiment, this measure was not shown in Figure 2.6. Maximum search distances in

millimeters and degrees are outlined as red dotted lines.

Figure 2.5: Distance measured (in millimeters) between points and associated error

measurements. Distance between the actual location of catheter tip during prior pullback step and

the actual location during current step (purple) represents the pullback distance. Distance

between the estimated location of catheter tip during prior pullback step and the actual location

during current step (green). Localization error: distance between actual location of catheter tip

and estimated location in the current step (blue). Maximum search distance is outlined as a red

dotted line. The extent of the search represents twice this value (-16 mm to +16 mm).

46

46

Figure 2.6: Distance measured (in degrees) and associated error measurements. Distance

between the estimated catheter tip orientation of previous pullback step and the current step true

orientation. Orientation error: distance (in degrees) between actual orientation of catheter tip and

estimated orientation of current iteration (blue). Maximum search distance (in degrees) is

outlined as a red dotted line. The extent of the search represents twice this value (-8 degrees to

+8 degrees).

A majority of the estimated prior position estimates remained within the maximum search

distance of the actual current location. During iteration 9 in the distance measurements and

iterations 8-10 of the angle measurements, the distance between the estimated prior and current

actual locations fell outside the maximum search distance. In these circumstances, it is not

possible for the algorithm to identify the current actual location given it is out of bounds of the

maximum search distance. The distance measure recovered by locating a different minimum in

the cost function that lay within the search radius and thus ‘recovered’; however the angle

measurements would likely continue to track in a false local minimum in further iterations in this

scenario. The angle estimates likely continued to follow false minima given the rotational

symmetry of the cardiac phantom about the x-axis (please refer to next section for details).

47

47

2.5 Discussion In our work, we present a software system for tracking the position and orientation of a 2D ICE

imaging surface within a previously acquired 3D CT volume. The algorithm begins with an

initial user-defined step, followed by a limited local search (32 mm translation and 16 degrees

rotation total) during each iteration of the process. Testing was performed using a custom-built

silicone cardiac phantom, and images acquired using a Toshiba CT scanner and 10 Fr Conavi

Foresight™ ICE imaging catheter. ICE imaging surface was tracked through the simulated right

ventricle of the 3D CT volume in ten successive pullback steps with an average localization error

for position of 4.1 mm. The localization error was measured by comparing the locations of the

ICE surface estimated by the algorithm and scanned location of the ICE catheter tip on CT. A

version of the CT of the cardiac phantom acquired with the ICE catheter removed was used to

test the performance of the algorithm.

The average localization error with respect to orientation for the ten pullback positions was 8.1

degrees, which was influenced by the rotational symmetry of the cardiac phantom. Rotations

about the x-axis particularly yielded similar RV cavity outlines, thus multiple false minima, as

demonstrated in Figure 2.7. Given the NGF distance measure assesses distance based on

geometric boundaries, the similar RV outlines caused the algorithm to erroneously “rotate” the

estimated orientation about the x-axis. This concept is well-illustrated by following the rotational

trajectory of the estimated surface during pullback in the top row of Figure 2.3. In more realistic

phantom or in-vivo anatomy, the rotation errors seen in this experiment will be much less likely

to occur due to the lack of rotational symmetry (due to irregular surfaces, and both near-field and

far-field additional structures, such as chordae or papillary muscles). Further work is required to

assess orientation estimate accuracy in models with less rotational symmetry.

48

48

Figure 2.7: Demonstration of rotational symmetry about the x-axis. In images to the far left and

right, the x, y, z position remained constant, however the rotation about the x-axis (or ‘pitch’)

varied between +10 degrees and -10 degrees. The 2D CT representations of the conical surfaces

extracted at each angle (20 degrees apart) are shown in the centre column. The centre column

demonstrates how geometric outlines at these varied angles would be difficult to discern without

any additional anatomic features, such as those found in detailed phantoms or in-vivo anatomy.

A key to maintaining processing efficiency is limiting the scope of the searches performed

during each iteration. Our experiment demonstrates that the algorithm is able to track the

location of the ICE imaging surface within a reasonable error range, however if the estimated

prior location (or orientation) of the search falls outside the maximum search distance of the

actual current location due to tracking a false local minimum, the algorithm may or may not

‘recover’. One possible solution to this dilemma is to increase the maximum search radius;

however, at a significant cost of processing efficiency. Strategies to potentially address this issue

will be discussed in Chapter 3. Further research is required to determine strategies to increase the

search area while maintaining processing efficiency.

For the purpose of our study, the search algorithm was developed in the MATLAB programming

environment, which provides a robust set of software development tools, however at a significant

cost of processing efficiency. The algorithm requires approximately 175,000 independent steps

(each requiring approximately 1 second), which are processed linearly in its current form,

totalling approximately 48 hours per iteration. Although this timeframe is unacceptable for its

49

49

intended use in structural heart disease, the algorithm will operate more efficiently using CPU

multithreading and GPU parallel processing techniques. For example, Xu, et al from our research

group developed a similar system for co-registering real-time with static MR images that was

accelerated by 139-fold using a graphics card with a modest only 192 CUDA cores. It is

estimated that using a contemporary high-end graphics workstation, such as one using four

NVIDIA Tesla V100 with a combined 20,480 CUDA cores could accelerate the process by over

14,800-fold. In this scenario, the rate of position updates would accelerate from the current rate

of 2 days per update to 12 seconds per update. Such a system would provide an acceptable

refresh speed for the purpose of tracking a live imaging surface during TAVI.

Using our system, an operator may perform ICE imaging during structural heart cases, such as

TAVI, with a better understanding of the three-dimensional anatomic context surrounding the

imaging catheter. For many early-experience ICE users, understanding the position and

orientation of the catheter tip represents a significant barrier to adoption. With further

experience, ICE allows the operator to visualize the valve prosthesis and surrounding soft tissue

simultaneously, as well as native valve deformation and Doppler information in real-time.

Operators will be better equipped to assess prosthesis positioning with respect to the native

valve, risk of coronary occlusion, and development of paravalvular leak, all in real-time and

without the use of additional nephrotoxic contrast injections. The localization error of 4.1 mm for

position represents an acceptable range for the purpose of guiding structural heart procedures.

For example, the most position-sensitive measure performed during TAVI relates to the risk of

coronary occlusion following valve deployment. In this case, the coronary origins usually

measure between 4-6 mm, and the borderline valve-to-coronary height is 10 mm. An estimate

within 4.1 mm will allow the operator to position their imaging surface within the vicinity of this

sensitive measure with subsequent fine adjustments as necessary. Using ICE imaging, the

operator is thus able to monitor the coronary origin in real-time as the prosthesis is deployed, and

the native leaflets are deformed in the direction of the ostium.

2.6 Conclusion The ICE-CT co-registration process described in this paper represents an accurate method for

determining the location of a 2D ICE imaging surface within a 3D CT volume. The software is

50

50

able to display the 2D ICE surface’s location in context of the prior CT volume. The algorithm

will potentially help structural heart operators appreciate the three-dimensional anatomic context

of the ICE imaging surface. Further work is required to improve processing efficiency and

increase the search area, including the use of multithreading and parallel processing techniques.

51

51

Chapter 3

Summary and Future Directions

In our work, we present a system for localization of an ICE imaging surface within a three-

dimensional CT volume. The localization algorithm begins with a user-defined step, which

provides the starting point for the position and orientation search. Subsequent steps involve

performing a series of image 2D-2D image co-registrations within a limited range of positions

and orientations (typically 32 mm and 16 degrees), and using the resulting ‘J-value’ (co-

registration metric determining the quality of the overlap) to determine the best estimate of the

position and orientation of the ICE image surface within the CT volume. The individual two-

dimensional co-registration steps are performed in MATLAB using the Flexible Algorithm for

Image Registration (FAIR) package. The algorithm uses three levels of multilevel image

representation as pre-processing steps. In this form of pre-processing, pixel values are averaged

in progressive steps such that coarse features of the images are aligned first, prior to alignment of

fine image features. The distance measure used in the algorithm to compare ICE and CT is

normalized gradient fields (NGF). NGF compares the locations of magnitude changes (ignoring

the absolute magnitude of the change), such as those found at major anatomic boundaries, which

are well preserved in both ICE and CT. Elastic regularization helps prevent tracking within false

minima by applying penalties to large transformations in a manner that simulates elastic potential

energy. Spline-based transformations are applied to the template during each iteration of the

algorithm, and compared to the reference image using NGF as a distance metric and an elastic

regularizer. Figure 2.2 in Chapter 2 shows a graphical representation of the 2D-2D portion of the

algorithm. The search for the ICE image surface within the CT volume is performed with 5

degrees of freedom (x/y/z translation, and pitch/yaw). In-plane rotation, the 6th degree of

freedom, is accounted for in the 2D-2D registration process. A sampling shortcut, known in our

work as progressive sampling, is performed to maintain processing efficiency of the 3D search.

Progressive sampling involves under-sampling the data set, and subsequently performing more

detailed sampling of data falling below as certain threshold value. Ultimately, the algorithm is

capable of tracking the location of an ICE imaging catheter within a silicone cardiac phantom

over ten iterations with an average localization error of 4.1 mm.

52

52

The average localization error for orientation was 8.1 degrees, which was similar to the extent of

the orientation search. Figure 2.6 demonstrates the error in orientation estimates related to

pullback position. The relatively high error value was heavily influenced by the rotational

symmetry of the cardiac phantom. Figure 2.3 demonstrates the estimated path of the ICE

imaging surface based on the algorithm (top row), compared to the actual trajectory confirmed

by CT images of the ICE catheter during pullback (bottom row). The predominant rotational

change throughout the estimated trajectory is rotation around the x-axis (or ‘pitch’). Figure 2.7

demonstrates how such rotational error can occur in the cardiac phantom used in this experiment,

which would greatly improve in the in-vivo circumstance given the extra anatomic details

present in both the near-field and far-field (such as irregular chamber boundaries, chordae,

papillary muscles, etc).

A future phantom experimental design to more accurately test the orientation error would

involve a phantom model with more realistic asymmetric anatomic features, such as right

ventricular papillary muscles, trabeculae, and a moderator band. A similar experiment with ten

pullback positions may be performed, and compared to the results of the experiment performed

in our work to assess the role of anatomic symmetry in orientation accuracy. An in-vivo

experimental design, such as a porcine animal study, with CT scans performed during each

pullback iteration for the purposes of position and orientation error assessment would also help

determine the role of irregular anatomic features in ICE surface localization accuracy.

In-Vivo Preclinical Studies

The next step in experimental testing will involve a preclinical in-vivo animal test, such as a

porcine model, to test the difference in performance of the algorithm between a simple cardiac

phantom and in-vivo anatomy similar to a human. A catheter pullback study may be performed

within a CT scanner, similar to the phantom study performed in our work, and would help clarify

whether the algorithm performs localization estimates more accurately in-vivo due to the

additional anatomic details present, or whether it is more prone to tracking into false minima.

The Conavi Foresight™ ICE catheter image software displays the ECG signal acquired during

imaging, and therefore allows the operator to time comparisons of ICE and CT images at

identical portions of the cardiac cycle. By timing the image co-registration to occur during early

or mid-diastole, it will reduce the potential error in co-registration due to cardiac motion. It will

53

53

also be necessary to obtain CT scans at each pullback location to confirm the true position and

orientation of the ICE catheter in order to determine the error values between the estimate and

true positions and orientations. Figure 3.1 demonstrates roughly co-aligned ICE and CT frames

from a porcine study performed in 2017 as an illustration of how these in-vivo images compare.

The data acquired during the study shown in the figure is not optimized for the purposes of using

our algorithm due to the irregular contrast opacification of the right heart chambers, and

inadequate opacification of the left heart chambers. This example illustrates how an edge-

sensitive distance measure, such as NGF may be affected by improper CT contrast

administration. It remains to be seen how the algorithm performs with image artifacts present (in

both ICE and CT), such as metal artifacts caused by cardiac pacemaker leads.

Figure 3.1: Single frame representation of ICE (left) and CT (right) surfaces demonstrating ICE

catheter in right atrium during porcine study. CT image created by extracting shallow cone in

three dimensions from CT volume in roughly similar orientation to ICE surface. ICE imaging

probe tip seen in centre of ICE image. ICE and CT representations of right atrium, aortic root

and pulmonary artery shown above. Data acquired during the above study is not optimized for

the purposes of using our algorithm due to the irregular contrast opacification of the right heart

chambers, and inadequate opacification of the left heart chambers. Future animal study will

require attention to appropriate contrast deliver and CT gating techniques in order to perform

accurate localization.

54

54

Addressing the Limited Search Area

A key consideration in improving the algorithm towards a clinically useful product is addressing

the limited search area. In the current version, it is possible for the algorithm to track an ICE

surface within a false local minimum, and the algorithm is unable to recover since the position

and/or orientation fall outside the maximum search distance. By expanding the search area, this

scenario is less likely; however, by increasing the search area, additional false local minima may

be uncovered beyond the perimeter of the previous maximum search distance. There are also

significant computer processing costs in increasing search area. In order to limit computational

costs, one may consider reducing final localization resolution, increase the angle searched in

exchange for a reduction in distance searched (or vice-versa), or modify the sampling ratio at

each step of the under-sampling strategy. Further work is required to better understand the

interplay between search area, search resolution, sampling ratios, and accuracy of the algorithm.

A strategy to increase the search area while maintaining processing efficiency is to perform

segmentation of the CT volume prior to the search in order to identify “vascular spaces”. This

method will ensure that the ICE catheter is tracked within vessels or cardiac chambers where it is

possible to be located, and ignoring non-vascular spaces in order to improve efficiency.

Improving Processing Efficiency: Parallel Processing Techniques

The algorithm designed in our work was developed in MATLAB using the FAIR image co-

registration package developed by Modersitzki,etal. Although both MATLAB and the FAIR

package provide robust development environments, there is a significant cost of processing

speed and efficiency during program runtime. In its current iteration, the algorithm requires

approximately 2 days to complete approximately 175,000 independent calculations required for

each localization step, each averaging approximately 1 second to complete. The timeframe

outlined is thus not compatible with imaging support during structural heart interventions. The

use of parallel processing methods afforded by graphics processing units (GPUs) has the

potential to significantly accelerate processes such as ours that require numerous small,

independent calculations. For example, a study performed by Xu, et al, from our research lab

were able to accelerate respiratory motion correction during co-registration of real-time with

static MR by an average of 139-fold using a GPU with a modest only 192 CUDA cores. Using a

55

55

contemporary high-end graphics workstation, such as one containing four NVIDIA Tesla V100

GPUs with a combined 20,480 CUDA cores could potentially accelerate the process by over

14,800-fold. It is therefore conceivable that combining implementation in a more efficient

programming environment, such as C#, and running the independent processes in parallel over a

high-end graphics workstation could reduce the time between iterations from 2 days to 12

seconds. Limiting the co-registration iterations to vascular-only territories may reduce data

processing by up to 50% further, thus providing an update speed of only 6 seconds. A refresh

rate in this timeframe would ultimately be compatible with assisting imaging guidance for

structural heart intervention.

Improving Processing Efficiency: Predictive Algorithms

A further step to consider in accelerating the algorithm to real-time capabilities for guiding

structural heart interventions is employing the use of predictive algorithms (such as Kalman

filtering and artificial neural networks). Kalman filtering, or linear quadratic estimation (LQE), is

an approach to predicting data patterns based on a prior data sample collected over time. The

process is used to estimate output variables of a control system prone to stochastic disturbances

cause by noise in the input dataset.88 The algorithm works by combining the known information

regarding the system dynamics learned by the training data, with the probabilistic information

regarding the noise. Its strength lies in the ability to predict past, present and future data, even

when the causes of the noise of the input signal is poorly understood. Kalman filtering has been

used extensively in aerospace navigation, automated driving, and real-time image processing.

For our purposes, Kalman filtering would be particularly useful for predicting the motion of the

ICE catheter. For example, the estimated position predicted by our algorithm will track the

movement of ICE catheter is within an average error of 4.1 mm. The movement of the ICE

catheter, combined with the noise introduced by our 4.1 mm error range will likely produce

‘jumps’ in position estimates. By using the Kalman filtering technique, the algorithm will predict

the location of next movement based on physical principles (which may be further enhanced by

the addition of hardware sensors), and uses our algorithm to ‘steer’ the updated predicted

position. The resulting updated position is based on the covariance between the path predicted by

prior data (along with physical principles) and the estimates produced by our algorithm.

56

56

Artificial neural networks (ANNs) are a subset of machine learning which make use of a series

of interconnected nodes, each known as an “artificial neuron”, to make predictions regarding the

outcome of a system. The connections between artificial neurons and the neurons themselves

carry as specific weighting factor, which is modified as a specific training data set is introduced.

ANNs are adept at predicting outcomes based on training data sets, without necessarily

understanding the underlying mechanism causing the outcome. In this fashion, they are

particularly well-suited for applications where prior behaviour of the system somewhat predicts

future events. For our purposes, ANNs may be used to predict the motion of the ICE catheter

with fewer 3D searches performed, as well as dictate the regions in 3D space to perform more

aggressive 2D-2D co-registration sampling in the 3D search portion of the algorithm. In

combination with our progressive sampling method, ANNs may substantially reduce the number

of 2D-2D co-registrations required in a search of 3D space required to accurately locate the

position and orientation of the ICE catheter.

Addressing Introduction of New Equipment and Tissue Deformation

In future versions of the algorithm developed in our work, it will be necessary to consider the

effect of tissue deformation and introduction of new equipment in the live imaging space. Given

the algorithm attempts to track the location of the live imaging surface within the pre-procedure

CT volume, it stands to reason that the introduction of foreign equipment during the procedure

may result in incorrect localization. This issue may be addressed using one or both of the

following strategies: 1) combination with a magnetic tracking system, such as the system

described by Luo, et al,62 and 2) computed simulation of introduction of equipment in the

imaging field. In the first example, our algorithm may be employed to make small position and

orientation adjustments for a magnetic tracking system. The advantage of this approach is that

using the position information from the MTS system, only require a small local search would be

required to update both position and orientation. A combined system would likely draw on the

strengths of each tracking technique without a significant delay in tracking if using the parallel

processing techniques described above. In the second example, retaining a software-only

solution may still be possible if the algorithm can accurately predict the appearance of certain

medical instruments in the imaging field. Given the imaging views used during TAVI are

predominantly in short-axis, and the equipment used are narrow catheters, the space occupied by

57

57

the equipment in the imaging field will be predominantly small, however simulating the

instrument and acoustic shadowing cast will remain a significant challenge.

Working Towards 3D-3D Image Co-Registration

The Conavi Foresight™ ICE catheter is capable of performing 3D ICE imaging by modifying

the rotation speed of the ultrasound transducer located at the distal tip. The mechanical rotating

element is a hexagonal disc that revolves about the longitudinal axis and acquires images

perpendicular to the catheter at speeds of approximately 20 Hz. The disc tilts to a ‘forward-

looking’ position at rotation speeds of 70 Hz (Figure 3.2).

Figure 3.2: 10-French Conavi Foresight™ ICE catheter (shown left) with proximal connector,

and handle for deflection. Imaging tip (shown right) rotates about the longitudinal axis as speeds

of roughly 20 Hz. By increasing the rotational speed to up to 70 Hz, the ultrasound transducer

tilts to a ‘forward-looking’ position, and acquires images of anatomy ahead of the catheter. The

‘forward-looking’ ICE image surface is a conical surface with a much steeper angle and smaller

radius compares to the ‘side-looking’ surface. Reproduced with permission from Conavi Medical

Inc®.

The catheter is capable of performing ‘sweeps’ in three-dimensional space by modifying the

rotational speed of the distal tip of the catheter over time. In a single sweep, the catheter is

capable of combining images at progressive angles to form a 3D ICE volume. This may be

performed in approximately one second. Using maximum intensity projection techniques, a

58

58

surface rendering of the three-dimensional volume is shown to the user based on pixel intensities

(shown in Figure 3.3).

Figure 3.3: Example of Conavi Foresight™ 3D intra-procedural imaging. By modifying the

rotational speed of the distal tip of the ICE catheter, the ultrasound moves to a progressively

more ‘forward-looking’ position. A resulting ‘sweep’ by modifying the rotational speed over

time produces a 3D image data set. The above is a maximum intensity projection of the catheter

performing a 3D sweep from within the right atrium. Reproduced with permission from Conavi

Medical Inc®.

A potential future direction for localization of the ICE catheter within a CT volume will depend

on co-registering maximum intensity projections of both ICE and CT in three dimensions. In this

case, the 2D localization may serve as a starting point, however with the capabilities of aligning

anatomic details in three dimensions, the catheter orientation results would be theoretically much

more accurate. 3D/3D co-registration between multiple imaging modalities has been developed

in prior work, and proven to be highly accurate.89

Final Thoughts Regarding Clinical Utility

59

59

The use of ICE imaging represents an emerging practice in the field of structural heart disease

intervention, including transaortic valve implantation (TAVI), and has the potential to improve

procedural guidance, as well as monitoring for procedural complications. ICE is particularly

useful in imaging the valve prosthesis location during deployment, monitoring the location of the

displaced native aortic leaflets in context of the coronary origins, and monitoring for the degree

of paravalvular leak post-valve deployment in real-time. Structural heart disease operators

(cardiologist or cardiac surgeon) would potentially make use of the localization algorithm

described in our work to track the anatomic location of an ICE imaging surface used during

TAVI within a previously acquired 3D CT volume. By demonstrating the location of the ICE

surface in this context, the operator will gain an enhanced understanding of the three-

dimensional anatomic context of the ICE imaging surface. A specific example of clinical

application of ICE imaging during TAVI that would benefit from an appreciation of three-

dimensional anatomic context provided by our algorithm is during deployment of a TAVI valve

with borderline low valve-to-coronary heights, typically in the 8-10 mm range. During the pre-

procedure planning CT study, the TAVI operator assesses the height of the coronary origins from

the native valve and the native valve leaflet lengths. During valve implantation, the native valve

leaflets are propped permanently in the ‘open’ position, and thus with low coronary heights there

is risk of causing iatrogenic coronary occlusion. This situation is easily predicted if the leaflet

lengths are longer than the estimated coronary heights. In this circumstance, the patient will

benefit from intentional leaflet laceration using a ‘Basilica’ procedure, however it is quite

technically difficult and introduces several new risks beyond TAVI itself. There are many

circumstances, however, where the leaflet lengths are ‘borderline’ to produce coronary occlusion

(ie. leaflet length is within 1-2 mm of coronary height), and require diligent intra-procedural

monitoring. In these circumstances, the operator may use ICE to monitor for coronary occlusion

during valve deployment. If the operator observes occlusion, the coronary origin may be stented

(which effectively displaces the valve away from the coronary origin) using a coronary wire left

in place prior to valve deployment as a precaution. In monitoring for this complication, the

operator will require a clear understanding of where the coronary origins lie with respect to the

current ICE imaging surface, and thus, a system that can provide this information within 4 mm

(which is roughly the diameter of the coronary origin) would be highly desirable. Additionally,

this information is available to the operator with minimal disruption to current TAVI workflow.

60

60

Final Thoughts

The potential combination of the use of parallel processing techniques, predictive algorithms,

machine learning, and computed simulation of procedural equipment will likely provide the

performance capabilities necessary to introduce the use of our algorithm into the hands of

structural heart operators for clinical application. Further pre-clinical and clinical experiments

will be required to confirm whether the addition of the localization algorithm described will

enhance the operators’ understanding the three-dimensional context of the current imaging plane.

In conclusion, if the algorithm performs as this author predicts in the real-world clinical setting,

it has the potential to improve procedural guidance during TAVI by introducing early

recognition of common serious complications, such as iatrogenic coronary occlusion, or

persisting paravalvular leak.

61

61

References 1. Asgar, A. W. et al. Structural Heart Disease Intervention: The Canadian Landscape. Can.

J. Cardiol. 33, 1197–1200 (2017).

2. Franke, J., Steinberg, D. H. & Sievert, H. Interventional treatment of structural heart disease. Minim. Invasive Ther. Allied Technol. 18, 110–121 (2009).

3. Steinberg, D. H., Staubach, S., Franke, J. & Sievert, H. Defining structural heart disease in the adult patient: current scope, inherent challenges and future directions. Eur. Hear. J. Suppl. 12, E2–E9 (2010).

4. Higgs, Z. C. J., Macafee, D. A. L., Braithwaite, B. D. & Maxwell-Armstrong, C. A. The Seldinger technique: 50 Years on. Lancet 366, 1407–1409 (2005).

5. Noble, S. & Roffi, M. Overcoming the Challenges of the Transfemoral Approach in Transcatheter Aortic Valve Implantation. Interv. Cardiol. Rev. 8, 131–134 (2013).

6. Bleakley, C. & Monaghan, M. J. The Pivotal Role of Imaging in TAVR Procedures. Curr. Cardiol. Rep. 20, (2018).

7. Carminati, M. et al. Role of imaging in interventions on structural heart disease. Expert Rev. Cardiovasc. Ther. 11, 1659–1676 (2013).

8. Cavalcante, J. L. & Schoenhagen, P. Role of cross-sectional imaging for structural heart disease interventions. Cardiol. Clin. 31, 467–478 (2013).

9. Cavalcante, J. L., Rodriguez, L. L., Kapadia, S., Tuzcu, E. M. & Stewart, W. J. Role of echocardiography in percutaneous mitral valve interventions. JACC. Cardiovasc. Imaging 5, 733–746 (2012).

10. Katz, W. E., Conrad Smith, A. J., Croc, F. W. & Cavalcante, J. L. Echocardiographic evaluation and guidance for MitraClip procedure. Cardiovasc. Diagn. Ther. 7, 616–632 (2017).

11. Lee, A. P. W. et al. Role of real time three-dimensional transesophageal echocardiography in guidance of interventional procedures in cardiology. Heart 96, 1485–1493 (2010).

12. Sengupta, P. P. & Khandheria, B. K. Transoesophageal echocardiography. Heart 91, 541–547 (2005).

13. Wunderlich, N. C., Kux, H., Kreidel, F., Birkemeyer, R. & Siegel, R. J. The Changing Paradigm in the Treatment of Structural Heart Disease and the Need for the Interventional Imaging Specialist. Interv. Cardiol. Rev. 11, 135–139 (2016).

14. Litmanovich, D. E. et al. Imaging in Transcatheter Aortic Valve Replacement (TAVR): role of the radiologist. Insights Imaging 5, 123–145 (2014).

15. Binder, R. K. et al. Prediction of Optimal Deployment Projection for Transcatheter Aortic

62

62

Valve Replacement. Circ. Cardiovasc. Interv. 5, 247–252 (2012).

16. Blanke, P., Schoepf, U. J. & Leipsic, J. A. CT in Transcatheter Aortic Valve Replacement. Radiology 269, 650–669 (2013).

17. Auffret, V. et al. Temporal Trends in Transcatheter Aortic Valve Replacement in France: FRANCE 2 to FRANCE TAVI. J. Am. Coll. Cardiol. 70, 42–55 (2017).

18. Akinseye, O. A., Jha, S. K. & Ibebuogu, U. N. Clinical outcomes of coronary occlusion following transcatheter aortic valve replacement: A systematic review. Cardiovasc. Revascularization Med. 19, 229–236 (2018).

19. Barbanti, M. Avoiding Coronary Occlusion and Root Rupture in TAVI – The Role of Pre-procedural Imaging and Prosthesis Selection. Interv. Cardiol. Rev. 10, 94–97 (2015).

20. Sultan, I., Siki, M., Wallen, T., Szeto, W. & Vallabhajosyula, P. Management of coronary obstruction following transcatheter aortic valve replacement. J. Card. Surg. 32, 777–781 (2017).

21. Carità, P. et al. Aortic stenosis : insights on pathogenesis and clinical implications. J. Geriatr. Cardiol. 13, 489–498 (2016).

22. Spaccarotella, C., Mongiardo, A. & Indolfi, C. Pathophysiology of Aortic Stenosis and Approach to Treatment With Percutaneous Valve Implantation. Circ. J. 75, 11–19 (2010).

23. Sverdlov, A. L. et al. Pathogenesis of aortic stenosis: not just a matter of wear and tear. Am J Cardiovasc Dis 1, 185–199 (2011).

24. Nkomo, V. T. et al. Burden of valvular heart diseases: a population-based study. Lancet 368, 1005–1011 (2006).

25. Mordi, I. & Tzemos, N. Bicuspid Aortic Valve Disease : A Comprehensive Review. Cardiol. Res. Pract. 2012, 1–7 (2012).

26. Roberts, W. C. & Ko, J. M. Frequency by decades of unicuspid, bicuspid, and tricuspid aortic valves in adults having isolated aortic valve replacement for aortic stenosis, with or without associated aortic regurgitation. Circulation 111, 920–925 (2005).

27. Horstkotte, D. & Loogen, F. The natural history of aortic valve stenosis. Eur. Heart J. 9 Suppl E, 57–64 (1988).

28. Iung, B. & Vahanian, A. Degenerative calcific aortic stenosis: a natural history. Heart 98, iv7-iv13 (2012).

29. Teppo, L. et al. Cancer patient survival--patterns, comparisons, trends--a population-based Cancer Registry study in Finland. Acta Oncol. (Madr). 38, 283–294 (1999).

30. Otto, C. M. Calcific aortic stenosis--time to look more closely at the valve. The New England journal of medicine 359, 1395–1398 (2008).

63

63

31. Grimard, B. H., Safford, R. E. & Burns, E. L. Aortic Stenosis: Diagnosis and Treatment. Am. Fam. Physician 93, 371–378 (2016).

32. Rijsterborgh, H. & Roelandt, J. Doppler assessment of aortic stenosis: Bernoulli revisited. Ultrasound Med. Biol. 13, 241–248 (1987).

33. Nishimura, R. A. et al. 2014 AHA / ACC Guideline for the Management of Patients With Valvular Heart Disease. J. Am. Coll. Cardiol. 63, e57-185 (2014).

34. Taniguchi, T. et al. Sudden Death in Patients With Severe Aortic Stenosis : Observations from the CURRENT AS Registry. J. Am. Heart Assoc. 7, (2018).

35. Edwards, W. S. & Smith, L. Aortic valve replacement with a subcoronary ball valve. Surg. Forum 9, 309–313 (1958).

36. Chikwe, J., Filsoufi, F. & Carpentier, A. F. Prosthetic valve selection for middle-aged patients with aortic stenosis. Nat. Rev. Cardiol. 7, 711–719 (2010).

37. Rodriguez-Gabella, T., Puri, R., Voisine, P., Rodés-Cabau, J. & Pibarot, P. Aortic Bioprosthetic Valve Durability: Incidence, Mechanisms, Predictors, and Management of Surgical and Transcatheter Valve Degeneration. J. Am. Coll. Cardiol. 70, 1013–1028 (2017).

38. Auensen, A. et al. Morbidity outcomes after surgical aortic valve replacement. Open Hear. 4, 1–11 (2017).

39. Johnston, L. E. et al. Outcomes for Low-Risk Surgical Aortic Valve Replacement: A Benchmark For Aortic Valve Technology. Ann. Thorac. Surg. 104, 1282–1288 (2018).

40. Thourani, V. H. et al. Outcomes of surgical aortic valve replacement in high-risk patients: A multiinstitutional study. Ann. Thorac. Surg. 91, 49–56 (2011).

41. Vassileva, C. M. et al. Evaluation of the society of thoracic surgeons online risk calculator for assessment of risk in patients presenting for aortic valve replacement after prior coronary artery bypass graft: An analysis using the STS Adult Cardiac Surgery Database. Ann. Thorac. Surg. 100, 2109–2116 (2015).

42. Leon, M. B. et al. Transcatheter or Surgical Aortic-Valve Replacement in Intermediate-Risk Patients. N. Engl. J. Med. 374, 1609–1620 (2016).

43. Smith, C. R. et al. Transcatheter versus Surgical Aortic-Valve Replacement in High-Risk Patients. N. Engl. J. Med. 364, 2187–2198 (2011).

44. Rougé, A. et al. Mid-term results of 150 TAVI comparing apical versus femoral approaches. J. Cardiothorac. Surg. 10, 1–10 (2015).

45. Leon, M. B. et al. Transcatheter Aortic-Valve Implantation for Aortic Stenosis in Patients Who Cannot Undergo Surgery. N. Engl. J. Med. 363, 1597–1607 (2010).

64

64

46. Mack, M. J. et al. Transcatheter Aortic-Valve Replacement with a Balloon-Expandable Valve in Low-Risk Patients. N. Engl. J. Med. (2019). doi:10.1056/NEJMoa1814052

47. Adams, D. H. et al. Transcatheter Aortic-Valve Replacement with a Self-Expanding Prosthesis. N. Engl. J. Med. 370, 1790–1798 (2014).

48. Reardon, M. J. et al. Surgical or Transcatheter Aortic-Valve Replacement in Intermediate-Risk Patients. N. Engl. J. Med. 376, 1321–1331 (2017).

49. Feldman, T. E. et al. Effect of Mechanically Expanded vs Self-Expanding Transcatheter Aortic Valve Replacement on Mortality and Major Adverse Clinical Events in High-Risk Patients With Aortic Stenosis: The REPRISE III Randomized Clinical Trial. J. Am. Med. Assoc. 319, 27–37 (2018).

50. Popma, J. J. et al. Transcatheter Aortic-Valve Replacement with a Self-Expanding Valve in Low-Risk Patients. N. Engl. J. Med. (2019). doi:10.1056/NEJMoa1816885

51. Kempfert, J., Linden, A. Van, Lehmkuhl, L., Rastan, A. J. & Holzhey, D. Aortic annulus sizing : echocardiographic versus computed tomography derived measurements in comparison with direct surgical sizing. Eur. J. Cardio-Thoracic Surg. 42, 627–633 (2012).

52. Feuchtner, G. et al. Prediction of Paravalvular Regurgitation After Transcatheter Aortic Valve Implantation by Computed Tomography : Value of Aortic Valve and Annular Calcification. Ann. Thorac. Surg. 96, 1574–1580 (2013).

53. Généreux, P. et al. Paravalvular leak after transcatheter aortic valve replacement: The new achilles’ heel? A comprehensive review of the literature. J. Am. Coll. Cardiol. 61, 1125–1136 (2013).

54. Barbanti, M. et al. Anatomical and procedural features associated with aortic root rupture during balloon-expandable transcatheter aortic valve replacement. Circulation 128, 244–253 (2013).

55. Kasel, A. M., Cassese, S., Leber, A. W., von Scheidt, W. & Kastrati, A. Fluoroscopy-Guided Aortic Root Imaging for TAVR. JACC Cardiovasc. Imaging 6, 274–275 (2013).

56. Petronio, A. S. et al. Optimal Implantation Depth and Adherence to Guidelines on Permanent Pacing to Improve the Results of Transcatheter Aortic Valve Replacement With the Medtronic CoreValve System. JACC Cardiovasc. Interv. 8, 837–846 (2015).

57. Guerios, E. E. et al. Second valve implantation for the treatment of a malpositioned transcatheter aortic valve. J. Invasive Cardiol. 24, 457–462 (2012).

58. Hayashida, K. et al. Transcatheter aortic valve implantation for patients with severe bicuspid aortic valve stenosis. Circ. Cardiovasc. Interv. 6, 284–291 (2013).

59. Bartel, T., Edris, A., Velik-salchner, C. & Mu, S. Intracardiac echocardiography for guidance of transcatheter aortic valve implantation under monitored sedation : a solution to a dilemma ? Eur. Heart J. 17, 1–8 (2016).

65

65

60. Vitulano, N. & Narducci, M. L. Technology update: intracardiac echocardiography – a review of the literature. Med. Devices Evid. Res. 8, 231–239 (2015).

61. Pibarot, P. et al. Incidence and Sequelae of Prosthesis-Patient Mismatch in Transcatheter Versus Surgical Valve Replacement in High-Risk Patients With Severe Aortic Stenosis. J. Am. Coll. Cardiol. 64, 1323–1334 (2014).

62. Luo, Z. et al. Magnetic navigation for thoracic aortic stent-graft deployment using ultrasound image guidance. IEEE Trans. Biomed. Eng. 60, 862–871 (2013).

63. Franz, A. M. et al. Electromagnetic tracking in medicine--a review of technology, validation, and applications. IEEE Trans. Med. Imaging 33, 1702–1725 (2014).

64. Peters, T. M. & Linte, C. A. Image-guided interventions and computer-integrated therapy: Quo vadis? Med. Image Anal. 33, 56–63 (2016).

65. Hu, Y. et al. MR to ultrasound registration for image-guided prostate interventions. Med. Image Anal. 16, 687–703 (2012).

66. Fitton, I. S. F. et al. Impact of Anatomical Location on Value of CT-PET Co-Registration for Delineation of Lung Tumors. Int. J. Radiat. Oncol. 70, 1403–1407 (2008).

67. Modersitzki, J. Numerical Methods for image registration. Oxford Univ. Press. New York (2004).

68. Kanekoa, S., Yutaka, S. & Satoru, I. Using selective correlation coefficient for robust image registration. Pattern Recognit. 19, 1165–1173 (2003).

69. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G. & Suetens, P. Multimodality image registration by maximization of mutual information. IEEE Trans. Med. Imaging 16, 187–198 (1997).

70. Pluim, J. P. W., Maintz, J. B. A. & Viergever, M. A. Mutual-information-based registration of medical images: a survey. IEEE Trans. Med. Imaging 22, 986–1004 (2003).

71. Modersitzki, J. FAIR: Flexible Algorithms for Image Registration. Soc. Ind. Appl. Math. 95–115 (2009).

72. Hodneland, E., Lundervold, A., Rorvik, J. & Munthe-Kaas, A. Z. Normalized gradient fields for nonlinear motion correction of DCE-MRI time series. Comput. Med. Imaging Graph. 38, 202–210 (2014).

73. Haber, E. & Modersitzki, J. Intensity gradient based registration and fusion of multi-modal images. Dep. Math. Comput. Sci. Emory Univ. (2004).

74. Reali, F., Priami, C. & Marchetti, L. Optimization Algorithms for Computational Systems Biology. Front. Appl. Math. Stat. 3, 1–12 (2017).

75. Fischer, B. & Modersitzki, J. Ill-posed medicine — an introduction to image registration

66

66

Ill-posed medicine — an introduction to image. Inverse Probl. 24, 1–16 (2008).

76. Broit, C. Optimal registration of deformed images. Ph.D. thesis, Comput. Inf. Sci. Univ. Pennsylvania, USA 121, 167 (1981).

77. Holden, M. A review of geometric transformations for nonrigid body registration. IEEE Trans. Med. Imaging 27, 111–128 (2008).

78. Huang, X. et al. Dynamic 3D ultrasound and MR image registration of the beating heart. Med. Image Comput. Comput. Assist. Interv. 8, 171–178 (2005).

79. Huang, X. et al. Dynamic 2D ultrasound and 3D CT image registration of the beating heart. IEEE Trans. Med. Imaging 28, 1179–1189 (2009).

80. Lang, P. et al. Surface-Based CT-TEE Registration of the Aortic Root. IEEE Trans. Biomed. Eng. 60, 3382–3390 (2013).

81. Li, F. P., Rajchl, M., White, J. A., Goela, A. & Peters, T. M. Ultrasound guidance for beating heart mitral valve repair augmented by synthetic dynamic CT. IEEE Trans. Med. Imaging 34, 2025–2035 (2015).

82. Li, F. P., Rajchl, M., Moore, J. & Peters, T. M. A mitral annulus tracking approach for navigation of off-pump beating heart mitral valve repair. Med. Phys. 42, 456–468 (2015).

83. Linte, C. A., Moore, J., Wedlake, C. & Peters, T. M. Evaluation of model-enhanced ultrasound-assisted interventional guidance in a cardiac phantom. IEEE Trans. Biomed. Eng. 57, 2209–2218 (2010).

84. Linte, C. A. et al. Accuracy considerations in image-guided cardiac interventions : experience and lessons learned. Int. J. Comput. Assist. Radiol. Surg. 7, 13–25 (2012).

85. Mcleod, A. J. et al. Phantom study of an ultrasound guidance system for transcatheter aortic valve implantation. Comput. Med. Imaging Graph. 50, 24–30 (2016).

86. Zhang, Q., Eagleson, R. & Peters, T. M. GPU-based visualization and synchronization of 4-D cardiac MR and ultrasound images. IEEE Trans. Inf. Technol. Biomed. 16, 878–890 (2012).

87. Zhang, Q., Eagleson, R. & Peters, T. M. Dynamic real-time 4D cardiac MDCT image display using GPU-accelerated volume rendering. Comput. Med. Imaging Graph. 33, 461–476 (2009).

88. Piovoso, M. & Laplante, P. A. Kalman filter recipes for real-time image processing. Real-Time Imaging 9, 433–439 (2003).

89. Li, G. et al. Accuracy of 3D volumetric image registration based on CT , MR and PET / CT phantom experiments. J. Appl. Clin. Med. Phys. 9, 17–36 (2008).