3-D Ultrasound-Guided Robotic Needle Steering in Biological...

IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 12, DECEMBER 2014 2899

3-D Ultrasound-Guided Robotic Needle Steeringin Biological Tissue

Troy K. Adebar∗, Ashley E. Fletcher, and Allison M. Okamura, Fellow, IEEE

Abstract—Robotic needle steering systems have the potential togreatly improve medical interventions, but they require new meth-ods for medical image guidance. Three-dimensional (3-D) ultra-sound is a widely available, low-cost imaging modality that maybe used to provide real-time feedback to needle steering robots.Unfortunately, the poor visibility of steerable needles in standardgrayscale ultrasound makes automatic segmentation of the nee-dles impractical. A new imaging approach is proposed, in whichhigh-frequency vibration of a steerable needle makes it visible inultrasound Doppler images. Experiments demonstrate that seg-mentation from this Doppler data is accurate to within 1–2 mm.An image-guided control algorithm that incorporates the segmen-tation data as feedback is also described. In experimental tests inex vivo bovine liver tissue, a robotic needle steering system imple-menting this control scheme was able to consistently steer a needletip to a simulated target with an average error of 1.57 mm. Imple-mentation of 3-D ultrasound-guided needle steering in biologicaltissue represents a significant step toward the clinical applicationof robotic needle steering.

Index Terms—Image-guided intervention, robotic needlesteering, ultrasound doppler, ultrasound imaging.

I. INTRODUCTION

P ERCUTANEOUS interventions involving the insertion ofneedles to anatomic targets within the body are applied in

the diagnosis and treatment of many diseases. We will describethe treatment of liver cancer, our intended application. Percu-taneous radiofrequency ablation (RFA) is a common treatmentoption for liver cancer patients who are not eligible for resec-tion or transplantation. In this procedure, long electrodes areinserted through the skin into the liver, and used to ablate can-cerous tissue. Current percutaneous RFA of liver cancer suffersfrom significant limitations [1]. Straight electrodes are unable toreach tumors in some portions of the liver because they areblocked by vasculature, lung, or other sensitive structures. Largetumors require multiple electrode insertions, with each puncturethrough the liver capsule increasing the risk of hemorrhage. Thisincreased risk can make patients with advanced liver disease orsevere comorbidities ineligible for the RFA procedure. Success-ful treatment requires a margin of cancer-free tissue to be ablatedaround the tumor, in order to reduce the likelihood of recurrence

Manuscript received January 6, 2014; revised June 20, 2014; accepted June20, 2014. Date of publication July 1, 2014; date of current version November18, 2014. Asterisk indicates corresponding author.

*T. K. Adebar is with the Department of Mechanical Engineering, StanfordUniversity, Stanford, CA 94035 USA (e-mail: [email protected]).

A. E. Fletcher and A. M. Okamura are with the Department ofMechanical Engineering, Stanford University, Stanford, CA 94035 USA(e-mail: [email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TBME.2014.2334309

Fig. 1. Bent-tip needle steering: (a) As a flexible needle with a bent tip isinserted into solid tissue, the lateral forces acting at the tip cause it to follow acurved path through the tissue. (b) Through a duty-cycle approach that combinesrotation and insertion, the radius of curvature of needle insertion paths can bevaried during steering.

[2]. For medium to large tumors, developing a sufficient ablativemargin is highly dependent on the clinician’s ability to locatethe electrode tip over multiple passes using medical imaging forguidance [3]. Similar limitations also affect other percutaneousprocedures, such as biopsy and brachytherapy, and treatment ofdiseases other than liver cancer.

Robotic needle steering enables the insertion of flexible nee-dles along controlled, curved, three-dimensional (3-D) pathsthrough tissue [4], [5]. Robotic needle steering offers the po-tential to improve percutaneous interventions with both longand short needle steering paths. Long needle steering paths canallow RFA electrodes to reach around sensitive structures to pre-viously unattainable targets, and to reach multiple tumors acrossthe liver with a single capsule wound. Short needle steering pathscan allow percutaneous RFA to be safely applied to larger tu-mors than previously possible, by creating precise ablations ofarbitrary shape. In this paper, our methods are inspired by thesecond, ablation-shaping application. Steering through a largerworkspace within the liver will require further development andintegration of path planning and control algorithms.

Multiple methods for achieving needle steering have beendeveloped, including manipulation of the needle base [4], [6],external manipulation of the surrounding tissue [7], and controlof precurved, overlapping cannula sections [8]. In this study,we focus on bent-tip needle steering. As depicted in Fig. 1, aflexible needle with a bent tip will naturally curve when insertedinto tissue, as a result of the lateral force acting on the needletip. By axially rotating the needle shaft, a robotic system canselect the direction of curvature, and thus steer a flexible needlethrough a 3-D workspace. These flexible steerable needles canalso be inserted along approximately straight paths by rotatingthe needle at a relatively high rate during insertion. A duty-cycleapproach [9], which combines intervals of pure insertion withintervals of rotation, can generate a range of curvatures.

0018-9294 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

2900 IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, VOL. 61, NO. 12, DECEMBER 2014

While robotic needle steering systems and algorithms havebeen discussed extensively in the literature, progression towardin vivo tests and patient studies has been limited [10]. Onemajor obstacle, and the focus of this paper, is the integration ofmedical imaging systems that can provide real-time feedbackof needle pose to robotic systems when steering in biologicaltissues.

II. PRIOR WORK

A. Control of Steerable Needles

There is significant prior art relevant to image-guided controlof needle steering, both for bent-tip steerable needles and otherapproaches. DiMaio and Salcudean formulated rigid needle in-sertion as a trajectory planning and control problem, defining aneedle manipulation Jacobian for base control [4]. Glozman andShoham [6] and Neubach and Shoham [11] used inverse kine-matics to control base-manipulation needle steering, with X-rayand ultrasound image feedback, respectively. Ko and Rodriguezy Baena used a model predictive control algorithm for trajectory-following control of a bioinspired actuated flexible needle [12].For bent-tip steering, a number of control approaches have beendescribed based on a nonholonomic model of steerable needlemotion in tissue [5]. Reed et al. demonstrated image-guidedneedle steering in a planar workspace [13], by combining a pla-nar motion planner [14], an image-guided controller [15], anda torsion compensator [16]. Wood et al. formulated trajectorytracking controllers based on duty cycling for 2-D [9] and 3-D[17] trajectories. Bernardes et al. combined closed-loop imagefeedback with intraoperative replanning to deal with obstaclesand dynamic workspaces [18]. Rucker et al. [19] used a slidingmode controller with feedback from an electromagnetic track-ing system. This control scheme has the advantage that it doesnot require any prior knowledge of needle curvature. Recently,Abayazid et al. demonstrated bent-tip needle control in gelatinand chicken breast using a robotically controlled ultrasoundtransducer to track the tip [20].

Without methods for imaging needles in biological tissues,experimental validation of many of these methods has been lim-ited to 2-D workspaces in transparent artificial tissues such asagar or polyvinyl chloride rubber, with optical cameras usedto simulate medical imaging. These artificial tissues provideonly limited validation, as their mechanical properties are sig-nificantly different from biological tissues [21]. Although somestudies have applied more realistic medical imaging methods[6], [11], [20], ultrasound imaging has only been applied intightly controlled experimental conditions, yielding best-caseneedle visibility.

B. Segmentation of Needles From Ultrasound Images

Computed tomography (CT) imaging, magnetic resonance(MR) imaging, and X-ray imaging could all potentially be ap-plied to intraoperative guidance of steerable needles, but each ofthese modalities has significant drawbacks. Our work uses 3-Dultrasound because of its many advantages. Ultrasound systemsimage in real time, do not produce ionizing radiation, and are

not strongly affected by the presence of metal objects such asneedles. Perhaps most importantly, ultrasound systems are in-expensive and portable, and are already standard equipment inexisting operating rooms and treatment suites.

Ultrasound does have disadvantages, such as low signal-to-noise ratio and poor image resolution compared to MR or CT.When imaging needles, ultrasound image quality is highly de-pendent on the angle between the ultrasound imaging plane andthe needle. At some angles, significant reverberation artifactsgenerated by the highly reflective needle surfaces may appear.At other angles, the needle may not be visible at all, as thereflected waves may be dispersed away from the transducer[22]. Even when the needle can be identified, various imagingartifacts can make it difficult to locate the tip of the needlealong its axis.

A large amount of prior art exists on the automatic segmenta-tion of needles from B-mode (grayscale) ultrasound data, withmuch of it focused on segmenting straight needles using a variantof the Hough transform. Although the Hough transform is com-putationally intensive, real-time segmentation of straight nee-dles has been demonstrated using variations on the algorithm,including dual-plane projections [23], coarse-fine sampling[24], [25], and parallel implementation on a graphics processingunit [26]. Other similar algorithms, such as the parallel integralprojection transform, have also been applied [27]. Similar meth-ods have been described for segmenting curved needles. Slightlycurved needles can be segmented using a standard Hough trans-form method [28], [29], while more strongly curved needles canbe segmented by including a parametrization of needle bending[28], [30], [31].

The underlying issue that makes automatic needle segmenta-tion a difficult problem is the poor visibility of needles in stan-dard B-mode ultrasound data. Several of the described methodshave shown promising results in favorable conditions. How-ever, image-guided robotic needle steering requires segmenta-tion algorithms that can process large 3-D datasets containinghighly curved, extremely thin (e.g., 0.5-mm diameter) needlesat undesirable orientations relative to the transducer. Rather thancontribute a new algorithm that attempts to overcome these chal-lenges, our aim in this study was to reduce the complexity ofthe segmentation task, by leveraging the needle steering robotto produce ultrasound image data that more clearly reveal theneedle.

C. Doppler-Based Segmentation

Ultrasound Doppler is a diagnostic technique that measuresfrequency shifts in reflected ultrasonic waves that result frommotion. Color and power Doppler imaging, which are availableon most modern ultrasound systems, are commonly applied tooverlay blood flow data on B-mode ultrasound. Vibrating solidobjects have also been shown to produce recognizable Dopplersignals [32]. This concept has been applied to localize straightneedles [33]–[35] and needle tips [36] in 2-D ultrasound, aswell as instruments in cardiac interventions [37], [38] and otherapplications [39], [40]. This technique has not previously beenapplied to segment highly curved needles.

ADEBAR et al.: 3-D ULTRASOUND-GUIDED ROBOTIC NEEDLE STEERING IN BIOLOGICAL TISSUE 2901

Fig. 2. Needle segmentation concept. A voice coil actuator (vibrator) vibratesthe needle, resulting in a Doppler response around the needle cross sectionin a 2-D ultrasound image. The needle is segmented by localizing the Dopplerresponse across the sweep of a mechanical 3-D ultrasound transducer, and fittinga curve through the resulting points.

Fig. 3. Block diagram of closed-loop control in ultrasound-guided needlesteering. Automatic segmentation of the needle from 3-D ultrasound data pro-vides feedback of needle configuration to a steering planner, which identifiesthe needle rotation and curvature necessary to reach a target point. A duty-cyclecontroller generates the velocity commands that allow motor control hardwareto move the needle tip along the desired path to the target.

In our segmentation method, low-amplitude high-frequencyvibration is applied to a steerable needle using an actuator lo-cated outside the tissue, and the resulting motion along thelength of the needle is detected using power Doppler ultrasoundimaging as depicted in Fig. 2. In an initial feasibility study [41],we found that our segmentation method was within 1–2 mmof manual segmentation from B-mode data. In that early work,piezoelectric diaphragms were spot welded to stainless steelwires, and used to generate vibration as a proof of concept. Inthis paper, we describe the design and validation of a new needlesteering robot, with an integrated voice coil actuator that clipsto the steerable needle and vibrates it during insertion. We alsodescribe a needle control algorithm that incorporates this newsegmentation method to allow closed-loop control of steerableneedles in simulated tasks. Since our imaging and control meth-ods are not constrained to translucent tissue simulants or 2-Dneedle paths, we are able to perform validation tests for 3-Dneedle steering in ex vivo animal tissues.

III. 3-D ULTRASOUND-GUIDED NEEDLE STEERING

Fig. 3 shows a block diagram of our closed-loop controlalgorithm for ultrasound guidance of needle steering. The goalof the closed-loop system is to steer the needle tip toward a

target point t defined in the ultrasound coordinate system. Inthe following sections, our segmentation algorithm, steeringplanner, and robot controller will be described in detail.

A. Needle Segmentation

Throughout the description of our algorithm, we will assumethat the needle is oriented roughly orthogonal to the imagingplane of a mechanical 3-D ultrasound transducer, as depictedin Fig. 2. An actuator is used to vibrate the needle, and theresulting motion of the needle and surrounding tissue produces aDoppler signal across the series of N Doppler images generatedby the ultrasound system. Segmentation proceeds in two stages,as depicted in Fig. 4. First, the cross section of the needle islocalized in each image through 2-D image processing. Second,the 3-D shape of the needle is reconstructed by fitting a curveto the series of points.

1) 2-D Image Processing: To remove Doppler noise, patcheswith less than 300 connected pixels are removed. (Forcomparison, the Doppler patch centered around the nee-dle typically has a connected area of 1000–2000 pixels.)After this preprocessing step, the image coordinates of the nee-dle cross section are estimated based on the remaining Dopplerresponse. As seen in the example image in Fig. 4, the Dopplerresponse from the vibration generally appears as an irregular,roughly circular patch centered on the needle, with a colorcomet tail artifact in the axial (yimage) direction [42]. The lat-eral (ximage) coordinate of the needle is, thus, estimated asthe centroid of the Doppler response. To account for the colorcomet tail, the axial image coordinate of the needle is esti-mated as the point that separates one-quarter of the integral ofthe Doppler response above, and three-quarters below. Thesefractions were selected empirically based on the results of ourfeasibility study [41].

We define the x, y, and z directions of the 3-D ultrasoundcoordinate system to be in the lateral, axial, and elevationaldirections, respectively. Based on the geometry of the ultra-sound transducer, the needle points are mapped from the 2-D image coordinate system to the 3-D ultrasound coordinatesystem, yielding the set of N needle points d(0) , . . . ,d(N ) .Each d(i) ∈ R3 holds the x-y-z coordinates of detectedneedle point i.

2) 3-D Shape Fitting: Based on the Doppler points identifiedin the previous step, we resolve the shape of the needle bydefining the x and y coordinates as functions of the z coordinateover the range of d(0) , . . . ,d(N ) . In this study, we use third-order polynomial functions

d(i)x = fx(d(i)

z ) = a0 + a1(d(i)z ) + a2(d(i)

z )2 + a3(d(i)z )3

d(i)y = fy (d(i)

z ) = b0 + b1(d(i)z ) + b2(d(i)

z )2 + b3(d(i)z )3

(1)

and solve for the vectors of polynomial coefficients a, b ∈ R4

that minimize the sum of the squared error over the measuredDoppler points. We selected this curve type because a third-orderpolynomial was sufficient to represent the range of curved nee-dle paths we considered in this study. Additionally, low-order


Fig. 4. Segmentation algorithm: (a) Doppler image. A 2-D ultrasound Doppler image from a series of N captured over a sweep of the imaging plane.(b) 2-D image processing. Doppler patches with fewer than 300 connected pixels are assumed to be noise and discarded (dotted lines). The remaining Dopplerdata (solid line) are used to estimate the position of the needle cross section (cross). In the lateral (xim age ) direction, the position is estimated as the centroid ofthe Doppler response. In the axial (yim age ) direction, the position is estimated as the lower bound of the region that contains one quarter of the integrated Dopplerintensity. (c) 3-D shape fitting. The 2-D-localized points are combined in 3-D based on position information from the 3-D ultrasound transducer. Third-orderpolynomial curves are fit to the identified points in order to define the needle’s 3-D shape over the series of N images.

Fig. 5. Steering planner: (a) The needle tip frame is located at the distal endof the needle. The frame is oriented with the ztip -axis tangent to the needle,and the ytip -axis pointing toward the center of curvature. The needle follows acircular path of radius ρ. (b) The steering planner determines the incrementalrotation δθ and the path radius ρ that will cause the needle tip p to reach thetarget point t.

polynomials have the advantage that they average out noisein the 2-D Doppler points, which is important in our method.Longer, more tortuous paths would require more sophisticatedcurves, possibly segmented polynomials constrained to be tan-gent at transition points.

B. Steering Planner

The steering planner interprets the segmentation results inorder to estimate the pose of the needle tip, described by aneedle tip frame. The steering planner then identifies a pathfrom the tip position to the target point. The radius of the pathρ and the incremental needle rotation δθ needed to follow thepath are output to the robot controller, which drives the motorsof the needle steering robot.

1) Estimating Tip Frame: We define the needle tip frame asshown in Fig. 5. The origin of the tip frame is located at point p,which is the distal end of the needle. For simplicity, we ignorethe local geometry of the bent tip. The tip frame is orientedwith the ztip -axis tangent to the needle at p, and the ytip -axispointing in the direction of curvature. The center of curvature

for the circular needle path c lies on the ytip -axis, at a distanceρ from p. In our model, rotating the base of the needle by δθcauses the needle tip frame to rotate by δθ about the ztip -axis,while inserting the needle causes the tip point p to move alonga circular path of radius ρ.

Based on the segmentation result, the tip point p is identifiedusing the z-coordinate of the most distal Doppler point and thepolynomial curve functions

p = [ px py pz ]T =[fx(d(0)

z ) fy (d(0)z ) d

(0)z

]T. (2)

While it should also be possible to estimate the tip frame orien-tation based on the needle segmentation result, for example, byfitting a circular arc to a curved section of the 3-D segmentationresult to define the steering direction, we have found that in thecurrent implementation the Doppler segmentation results are toonoisy to allow reliable estimation of the tip frame orientation.Instead, we apply a small angle approximation, and assume thatthe ztip -axis remains close to the z-axis of the 3-D ultrasoundcoordinate system throughout steering. The orientation of thetip frame can, thus, be specified completely by a rotation of θabout the z-axis. We make rotation angle θ a system variable,and update it after each iteration of the steering planner

θi+1 = θi + δθi. (3)

This approach ignores torsional deflection of the steerable nee-dle as result of friction between the needle and tissue. Whilesubstantial torsional deflection has been demonstrated in artifi-cial tissues, in biological tissues the needle tip rotation has beenshown to follow the base rotation to within a few degrees [16].

2) Path Planning: At each iteration, the steering planneridentifies the simplest possible path to the target point t basedon the estimated tip pose. As shown in Fig. 5, this path is thecircular arc that is tangent to the estimated ztip -axis, and passesthrough t. Unlike other needle control approaches described inSection II [19], [20], our replanning approach does not attemptto minimize total needle rotation, and requires an estimate of


Fig. 6. Experimental setup. Needle steering robot consisting of actuated linear slide, rotation stage, and vibration module (red arrows indicate actuation); ex vivobovine liver tissue; 3-D ultrasound transducer.

maximum needle curvature. However, it is sufficient to demon-strate accurate closed-loop steering for small paths.

C. Robot Controller

The robot controller consists of two components. A duty-cycle controller generates motor velocity commands based onthe desired needle rotation and curvature as output by the steer-ing planner. Motor control hardware drives the motors of theneedle steering robot based on the velocity commands.

1) Duty-Cycle Controller: The goal of the duty-cycle con-troller is to determine the desired velocities for the rotation andinsertion stages, krot and kins . It is first necessary to calculatethe duty-cycle fraction, DC, based on the desired radius of cur-vature. Assuming a known minimum radius of curvature for thespecific needle-tissue pair, DC can be calculated as

DC = 1.0 − ρmin

ρ. (4)

The insertion velocity kins is set to a constant value duringsteering. To achieve the duty-cycle effect, the rotation stagealternates between constant velocity krot = krot,max and zerovelocity krot = 0 with the needle held at angle θ. The ratio oftime spent rotating (Trot) to time holding at angle θ (Thold ) isdetermined from the duty-cycle ratio DC

DC =Trot

Trot + Thold. (5)

Rotation time Trot is constant and equal to the time requiredto complete a full rotation. Hold time Thold is, thus, varied toachieve the desired ratio DC.

2) Control Hardware: There are many possible hardwareimplementations that could be used to drive the motors ofthe needle steering robot. In the system we use for exper-imental validation in Section IV, the rotation and insertionmotors are each driven by integrated motion control systems.These systems accept velocity commands (i.e., krot , kins) anddrive the motors using proportional-integral (PI) control. In oursystem, PI control constants were tuned using manufacturersupplied software.

Fig. 7. Vibration module. A voice coil actuator is attached to the steerableneedle using a 3-D-printed clip, and used to vibrate the needle vertically (in andout of the page).

IV. EXPERIMENTAL VALIDATION

In this section, we describe experimental validationof our algorithms for closed-loop 3-D ultrasound-guidedneedle steering.

A. Apparatus

1) Ultrasound Imaging: A SonixMDP ultrasound console(Ultrasonix Medical Corp., Richmond, BC, Canada) with aconvex mechanical 3-D transducer (4DC7-3/40) was used forimaging. Custom software incorporating the Ultrasonix researchSDK package was used to control imaging parameters and cap-ture images. Power Doppler imaging mode was selected overcolor Doppler imaging because of the lack of aliasing and re-duced sensitivity to imaging angle. Pulse repetition frequency(PRF) was set to 1428 Hz, as this setting yielded the best resultsin our initial feasibility study [41]. Wall motion filter was set tomaximum in order to minimize Doppler artifacts resulting fromthe motion of the imaging plane. Each sweep consisted of 61scan-converted 2-D images, captured at angular increments ofapproximately 0.7◦.

2) Needle Steering Robot: Fig. 6 shows the needle steeringrobot used to validate our control method. Similar to previ-ously described needle steering robots [43], our system has twoactive degrees of freedom (DOF) that control needle insertionand needle rotation. Unlike previous needle steering robots,our system incorporates a vibration module that generates the


high-frequency motion necessary to visualize the needle usingultrasound Doppler.

The rotational DOF is actuated by a geared DC motor(A-max 22-110160; Maxon Motor, Sachseln, Switzerland)that is connected directly to the needle through a pin-vise clamp. The insertion DOF is actuated by a DC motor(GM9234S016; Ametek, Berwyn, PA, USA) that drives a lin-ear slide (SPMA2524W4; VelMex, Bloomfield, NY, USA).Both the rotation DOF and insertion DOF are driven by inte-grated motion controllers (MCDC3006S; Faulhaber, Schnaich,Germany). The vibration module, shown in Fig. 7, consists ofa voice coil actuator (HIAX19C01-8; HiWave, Little Gransden,U.K.) that is driven by a transistor circuit at a user-variable fre-quency of 400, 600 or 800 Hz. These frequencies are roughlycentered around the peak output point of the voice coil actu-ator and, thus, produce the strongest responses in the powerDoppler image data. To produce the maximum vibration alongthe needle, the vibration module was designed to be as close aspossible to the target tissue, and is attached to the distal end ofthe needle steering robot. The needle is mated to the actuatorusing a 3-D-printed plastic clip.

3) Needles: Solid Nitinol wires 0.38 mm (0.015 inches),0.48 mm (0.019 inches) and 0.58 mm (0.023 inches) in diameterwere used as needles. The needles had 45◦ beveled tips and 4.5-mm distal sections bent 35◦ off axis.

4) Tissue Simulant: Ex vivo bovine liver tissue obtainedfresh from a local butcher was used as the tissue simulant inall testing.

B. Segmentation Accuracy

As mentioned in Section II, we previously described a feasi-bility study in which we measured the accuracy of our Dopplersegmentation method using piezoelectric elements welded tostainless steel wires [41]. In that study, we examined the sensi-tivity of the method to tissue composition, vibration frequency,and Doppler PRF, and found that only PRF greatly affected theaccuracy of segmentation. Based on comparison with manualsegmentation, we found the average error to be 1–2 mm acrossmost conditions.

In this study, we evaluated the accuracy of segmentation whenusing a needle steering robot with an integrated actuator to vi-brate flexible Nitinol needles. We again considered the impactof vibration frequency, although at a lower range than our pre-vious experiment because of the lower center frequency of thevoice coil actuator compared to the previous piezoelectric actu-ator. We also considered the impact of needle curvature, needlediameter, and position of the needle in the lateral image direc-tion. Two needle curvatures (straight and maximum curvature),two lateral positions (image center and lateral edge), three nee-dle diameters (0.38, 0.48, and 0.58 mm), and three vibrationfrequencies (400, 600, and 800 Hz) were tested. Each parame-ter was varied individually around the base case of a straight,0.48-mm needle centered in the image and vibrated at 600 Hz.

Three separate insertions and scans were performed for eachtest condition. Segmentation accuracy was evaluated by com-parison with a reference manual segmentation of the needle

from B-mode data, within the native image planes of the 3-Dultrasound transducer. To create the reference data, the center ofthe needle was manually selected in each 2-D B-mode image inwhich it was visible. The needle was visible in approximately50% of the B-mode images captured over most sweeps.

Given K manually selected needle points m(0) , . . . ,m(K ) ,we defined the segmentation error for the ith point

e(i) =

√(m

(i)x − fx

(m

(i)z

))2+

(m

(i)y − fy

(m

(i)z

))2.

(6)For this segmentation accuracy testing, the Doppler segmenta-tion method and the reference manual segmentation were bothimplemented in MATLAB.

The precision of the reference data was quantified by measur-ing variability between repeated manual segmentations of thesame B-mode volumes. Although this does not measure the ab-solute accuracy of the reference data, it provides an indication ofmanual segmentation error. Across four repeated segmentationseach for eight needle scans (four of straight needles and four ofcurved needles), the standard distance deviation

Sxy =

√√√√

M∑

i=1

(d(i)MC)2

M − 2(7)

was calculated, where d(i)MC is the distance in the image plane be-

tween the ith manually selected needle point and the correspond-ing mean center, and M is the total number of comparison points.With M = 684 data points, we found the standard distancedeviation for the manual segmentation to be Sxy = 0.23 mm.

C. Closed-Loop Steering

We validated the needle segmentation algorithm and steeringcontroller in a series of needle steering tests. The algorithmsdescribed above were implemented in C++ using the ITK andVTK libraries [44] for image analysis. Six insertion tests wereperformed, using 0.48-mm needles with bent tips as describedabove. In each test, a brachytherapy sheath was used to inserta cylindrical stainless steel bead into a liver tissue specimen, inorder to generate a target. The beads were 3 mm in diameter and5 mm long, and were oriented for maximum visibility in the 3-DB-mode images (i.e., with the long axis of the beads approxi-mately perpendicular to the axial direction of the ultrasound) toenable the manual reference segmentation. The 3-D ultrasoundtransducer was applied to the tissue and oriented so that thetarget was near the boundary of the ultrasound volume in the z(elevational) direction. The steerable needle was then inserteduntil it was just visible at the opposite end of the ultrasoundvolume. The initial position of the steerable needle tip p relativeto the target t was varied within a rectangular workspace withapproximate dimensions of 20 × 20 × 30 mm. This workspaceis representative of ablation shaping for medium-sized liver tu-mors (between 30 and 50 mm in diameter) using a single RFelectrode, assuming a rigid needle or other introducer was usedto initially place the steerable needle near the target. The dimen-sions of the needle steering path for each closed-loop steering


(a) (b)

Fig. 8. Two-dimensional ultrasound images taken from volumetric sweeps of steerable needles inserted into ex vivo liver: (a) B-mode ultrasound image used formanual segmentation. The needle cross section is indicated. (b) Corresponding power Doppler ultrasound image captured during needle vibration. The Dopplerresponse surrounds the needle cross section.

test are listed in Section V. The initial orientation of the steer-able needle tip was such that the tip frame was aligned with theultrasound coordinate system (i.e., θ = 0◦). After these initialinsertion steps, a 3-D ultrasound volume was captured, and thebead was manually segmented to yield the position of the targetin the 3-D ultrasound frame. Point t was placed on the surfaceof the bead closest to the starting position of the needle tip. Theneedle was then automatically steered toward the target untilthe system detected that the needle tip had reached the targetdepth in the z direction, at which point insertion was stopped.Targeting error was measured as the 3-D distance between thetarget point t and the needle tip p at the end of the test.

In this study, the control algorithm depicted in Fig. 3 wasimplemented in discrete increments. The control inputs of ro-tation and duty cycle were held constant over increments ofinsertion. At the end of each increment, insertion was stopped,and the needle was vibrated, scanned, and segmented automati-cally. The steering controller was then applied to determine therotation and duty-cycle inputs for the following increment. Rela-tively large 5-mm insertion increments were selected for testingto reduce deformation of the tissue due to needle stiction. Inser-tion velocity kins was set to yield a linear needle insertion rateof 100 mm/min. Maximum rotation velocity krot,max was set toyield a rotary needle velocity of 60 RPM.

Minimum radius of curvature ρmin for the 0.48-mm needlein ex vivo bovine liver tissue was measured in a separate initialexperiment. The steerable needle was inserted into the liverwithout rotation, scanned and segmented, and a circular arc wasfit to the segmentation results. Over four repetitions, the averageradius of curvature was found to be 51.4 mm, which is within therange of curvature values previously reported in ex vivo tissue[10]. This was the value of ρmin used by the steering planner tocalculate the duty cycle ratio DC based on the desired radius ofcurvature.

V. RESULTS AND DISCUSSION

Fig. 8 shows example 2-D ultrasound images taken fromvolumetric sweeps of steerable needles in ex vivo liver. Both B-mode and power Doppler images are shown. The needle cross

section was visible in approximately 50% of the B-mode im-ages captured over most sweeps. As shown in the figure, thehigh-frequency vibration of the needle causes a power Dopplerresponse centered around the needle cross section.

Fig. 9 shows the 3-D Doppler data generated by a vibrat-ing needle, along with the corresponding Doppler centroids andthe best-fit polynomial curve. Volumetric frame rate using theSonixMDP system was 0.2 Hz for power Doppler data at animaging depth of 50 mm. Software run-times were approxi-mately 60 ms per image for the 2-D image processing algorithmand 15 ms per sweep for the steering planner, when implementedon the SonixMDP’s integrated processor.

A. Segmentation Accuracy

Fig. 10 summarizes the results of the segmentation accuracytests. Across all tests, the maximum segmentation error was3.18 mm, the minimum segmentation error was 0.13 mm, themean segmentation error was 1.24 mm, and the standard de-viation of the segmentation error was 0.57 mm. In the basecase tests (straight, 0.48-mm needle centered in the image andvibrated at 600 Hz), the mean segmentation error was 0.92mm and the standard deviation of the segmentation error was0.93 mm.

In the needle curvature test, curved and straight needles hadsimilar segmentation errors, with somewhat higher errors inthe curved configuration. The mean error was 0.85 mm forstraight needles, and 1.36 mm for curved needles. We con-clude that for transducers initially oriented as shown in Fig. 2,our segmentation method is suitable for the range of needlegeometries that can be achieved by bent-tip steering in bio-logical tissue (with ρmin ≈ 50 mm). The current implemen-tation does fail if the steerable needle is exactly parallel tothe imaging plane, thus presenting a linear rather than circularcross section. However, this can be avoided by placing the 3-Dtransducer so the needle is initially approximately perpendic-ular to the imaging plane. Applying the Doppler segmentationtechnique without constraint on the transducer orientation, orwith other 3-D ultrasound implementations—such as freehand


Fig. 9. Example segmentation of needle using Doppler method: (a) Power Doppler response around the needle in each image. (b) Reconstructed needle shape(blue curve), fit through the centroids of the Doppler data in each image (red crosses).

Fig. 10. Segmentation accuracy results. Two needle configurations (straight, curved), two image positions (center, lateral), three needle diameters (0.38, 0.48,0.58 mm), and three vibration frequencies (400, 600, 800 Hz) were tested. For each group, red line indicates median error, blue box indicates 25th and 75thpercentile, and whiskers indicate minimum and maximum error. Number of data points for each group is also indicated.

tracked 3-D ultrasound—might require modification of theimage analysis method.

Needle diameter had a significant effect on segmentation ac-curacy. Mean error was 0.85 mm for 0.48-mm needles, versus1.98 mm for 0.38-mm needles, and 1.68 mm for 0.58-mm nee-dles. The significant change in mean error for the 0.48-mmneedles is surprising given the relatively small change in needlediameter. The complexity of the wave mechanics that governthe transmission of vibrations along a flexible needle in vis-cous tissue makes it difficult to theorize why needle diameterappears to be an important factor. This issue is worthy of fu-ture study for two reasons. First, because improved understand-ing of the motion around the vibrating needle might allow fornew Doppler processing algorithms that specifically target thegenerated motion. Second, because specific needle geometriesmay be required for other clinical applications (for example,brachytherapy systems would likely require hollow needles).

Lateral image position and vibration frequency did not havesignificant impacts on segmentation accuracy. In the lateral po-sition test, the mean error was 0.85 mm for needles centeredin the image, and 1.24 mm for needles on the lateral edge. Inthe vibration frequency test, the mean error was 1.16 mm for400-Hz vibration, 0.85 mm for 600-Hz vibration, and 1.14 mm

TABLE ICLOSED-LOOP NEEDLE STEERING RESULTS

Test N in c δx (mm) δy (mm) δz (mm) e f in a l (mm)

1 6 1.99 10.74 29.09 1.572 7 8.42 7.15 29.16 1.733 6 7.81 2.63 31.44 1.274 6 8.04 6.81 31.78 2.085 6 1.75 5.06 26.46 1.896 6 6.13 3.83 32.26 0.86

for 800-Hz vibration. In the case of vibration frequency, the in-sensitivity is in agreement with our previous study [41], wherevarying vibration frequency over a much larger range did notsignificantly affect segmentation accuracy.

B. Closed-Loop Steering

Table I lists results from six successful closed-loop needlesteering tests. For each test, the number of insertion increments(Ninc), the x-y-z dimensions of the complete needle steeringpath (δx, δy, and δz), and the final error in tip placement relativeto the target (efinal) are listed. Mean tip placement error was


Fig. 11. Closed-loop needle steering results. Orthogonal views of successive incremental scans during needle steering in ex vivo tissue, with the output from thesteering planner after each scan below. The needle base was inserted 5 mm between each scan, with rotation and duty cycle determined by the steering planner.Error in placing the tip at the target bead was 0.86 mm after the final insertion.

1.57 mm over the six tests, which is close to the mean error inFig. 10 for that configuration.

Fig. 11 shows the needle shapes reconstructed during incre-mental closed-loop steering in Test 6. The ytip -axis was initiallyaligned with the y-axis of the ultrasound coordinate system.After Scan 1, the steering planner compensated by rotating theytip -axis toward the target (δθ = −58.7◦). The steering plannergave smaller corrections based on Scans 2 to 5, as a result ofdeviation from the steering model and noise in the measurementof point p. After Scan 6, when the needle tip was within sev-eral millimeters of the target, the steering planner output a largeunnecessary rotation correction (δθ = −97.7◦), but over such asmall distance it had little effect. Final tip placement error efinalwas 0.86 mm for this test.

In addition to the six tests listed in Table I, there were severaltests that failed as a result of poor needle steering behavior.Bent-tip needle steering requires a homogeneous solid medium.If the steerable needle tip impacts an obstacle which it does notpierce through, such as a bone or blood vessel, the path of theneedle may deviate greatly from what is expected. Fig. 12(a)shows the reconstructed needle shapes from one such failedtest. In this test, the needle appeared to impact one of severalsmall vessels that could be visualized in ultrasound before thefinal scan. Fig. 12(b) shows examples of these small vessels ina section of liver specimen.

C. DiscussionThe implementations we have described are capable of seg-

menting a curved needle with average error of 1–2 mm, andsteering a needle to a target with an average error of 1.57 mm.This is roughly equivalent to manual targeting error with medicalimage guidance, which has been shown to be 2 mm or higheron average [45]–[47]. Studies on percutaneous RFA of livercancer generally suggest that an ablative margin of 5–10 mmaround a tumor is necessary to prevent recurrence of cancer [1]–[3]. However, recent work has reported that ablative margins assmall as 3 mm are associated with a lower rate of local tumorprogression [48]. This suggests that a targeting error of 2 mmmay be tolerable. Overall, the experimental results described in

this paper show that our segmentation and control algorithmsapproach the accuracy and consistency necessary for a practicalclinical application such as RFA of liver cancer. Still, there areseveral areas for improvement.

So far, we have applied our segmentation and control algo-rithms to sequences of incremental insertions, rather than to con-stant velocity insertions. A delay of several seconds after eachincremental insertion would be undesirable in a clinical setting,although perhaps not prohibitive. In our current experimentalsetting, we were limited by the capture rate of our 3-D Dopplersystem. (Doppler imaging frame rates are generally slower thanB-mode imaging frame rates.) There are promising options forimproving the speed of both the image capture and image pro-cessing. For example, since our segmentation algorithm can beapplied to the raw 2-D images, each individual frame of datacould be processed immediately upon capture by the sweepingultrasound array, rather than after capturing a complete volume.This would allow image capture and segmentation to take placeconcurrently, greatly reducing overall imaging time.

Continuous 3-D ultrasound-guided steering using our methodwould also require continuous vibration of the needle for theDoppler segmentation. This raises the issue of the effect ofvibration on steering. While we expect the impact to be smallbased on the magnitude of the vibration (the estimated amplitudeof vibration was less than 0.5 mm), the mechanical interactionbetween the needle and deformable biological tissue is complexand may be sensitive to this perturbation. Although initial anec-dotal testing has not revealed any impact, these issues requirefurther study.

Apart from increasing the speed of image capture and seg-mentation, there are several other directions for improvingthe Doppler segmentation method. Information on mechani-cal properties of the needle, such as minimum bending radius,could be used to improve the initial 2-D segmentation, for ex-ample by limiting the search region for the needle cross sec-tion based on the maximum possible needle curvature. Com-bining data from multiple ultrasound scan modes (e.g., Dopplerdata, B-mode data, RF scanline data, and elastography data)could also potentially improve the accuracy and consistencyof segmentation.


Fig. 12. Closed-loop needle steering results. Example of failed needle steering: (a) Orthogonal views of successive incremental scans during needle steering inex vivo tissue, with the output from the steering planner after each scan below. The needle base was inserted 5 mm between each scan, with rotational orientationand duty-cycle setting determined by the steering planner outputs. Error in placing the tip at the target bead was 8.14 mm after the final insertion. The needle failedto reach the target due to interference from internal vessels that could be visualized in the ultrasound images. (b) Section of tissue sample with example vesselsand approximate direction of needle insertion indicated.

In our current implementation, the orientation of the tip frameis derived from segmentation results using a small angle approx-imation. While this approximation is reasonable for the range ofneedle steering paths described in Section V.B, which are repre-sentative of ablation shaping around a tumor, the approximationwould fail for longer paths steering through a large workspacein the liver. For this application, we plan to implement an un-scented Kalman filter or other similar estimation scheme. Thiswill allow position feedback to be combined with a kinematicmodel of needle tip motion, in order to estimate the completetip pose.

Needle vibration is currently achieved by an actuator con-nected to the proximal end of the steerable needle. In this im-plementation, the Doppler response decreases moving alongthe needle toward the tip, as the vibrations are damped outby the surrounding tissue. For very long insertion paths, ormore viscous tissues, the amplitude of vibration at the tipmight be too small to allow a reliable Doppler segmentation.We are currently exploring other actuation schemes that vibratethe needle tip directly, as in [36] or [39], although integratingthese actuators into submillimeter steerable needles is a difficultpractical problem.

With vibration of the needle, the added potential for tissuedamage is an issue that must be considered carefully. Unfor-tunately, both the tissue strain around the vibrating shaft andthe displacement of the sharp needle tip are difficult to measureexperimentally, and it was not possible to quantify these factorsin the current work. Histological analysis after steering in anin vivo model, as in [10], would likely be required to evaluatetissue damage. As stated earlier, the displacement of the needleshaft is in the submillimeter range at the needle base, and de-creases along the needle shaft due to tissue damping. While wethus expect the resulting tissue damage to be minimal, this issueis again worthy of further study.

It should be noted that the duty-cycling approach we applyto control steerable needle curvature has largely been validated

in homogeneous artificial tissues, although several examples ofduty cycling in biological tissues have been reported [49]–[51].Recent work [50] suggests that the linear relationship betweenduty cycle and curvature described in (4) may oversimplifysteerable needle behavior in heterogeneous biological tissue.Fortunately, the closed-loop structure of our steering algorithmcompensates for deviation from the expected relationship, andfor variation in ρmin . Alternatively, a control scheme that doesnot rely on duty cycling, such as the sliding mode control de-scribed in [19], could be employed.

Design of a practical needle steering robot that is appropri-ate for in vivo testing is another important direction for futureresearch directed toward clinical needle steering. Since the vastmajority of prior needle steering methods have been evaluatedin artificial or ex vivo tissues, needle steering robots describedto date have not generally been appropriate for actual clinicalenvironments. We will modify our robot design to delineate dis-posable components, sterilizable components, and componentsthat must be draped or otherwise isolated. While our segmenta-tion approach introduces the voice coil actuator and connectingclip, these components should not greatly complicate steriliza-tion since they are both internal to the robot and can easily bemade disposable.

VI. CONCLUSION

We have demonstrated a method for 3-D ultrasound guidanceof robotic needle steering in biological tissue. The use of ul-trasound Doppler imaging, in combination with high-frequencylow-amplitude vibration of the steerable needle, greatly sim-plifies a challenging segmentation problem. Using the Dopplermethod, unsophisticated image processing methods can local-ize the needle with run-times of approximately 60 ms per im-age, and error on the order of 1–2 mm for relevant needleconfigurations. This segmentation algorithm is robust to curva-ture, image orientation, and vibration frequency. Experiments in


ex vivo bovine liver tissue demonstrate that our method allows aneedle steering robot to reach a simulated target with an averageerror below 2 mm.

In our future work, we will continue to refine our methods forultrasound-image guidance, with the goal being implementationof a clinical needle steering system for RFA of liver cancer. Thisserves as a first step toward deploying needle steering systemsin a variety of organ systems and clinical applications.

ACKNOWLEDGMENT

The authors wish to acknowledge the contribution of T. Jiangto the construction of the needle steering robot.

REFERENCES

[1] D. A. Gervais, N. Goldberg, D. B. Brown, M. C. Soulen, S. F. Millward,and D. K. Rajan, “Society of interventional radiology position statementon percutaneous radiofrequency ablation for the treatment of liver tumors,”J. Vasc. Interv. Radiol., vol. 20, no. 1, pp. 3–8, 2009.

[2] Y. S. Kim, H. Rhim, O. K. Cho, B. H. Koh, and Y. Kim, “Intrahep-atic recurrence after percutaneous radiofrequency ablation of hepatocel-lular carcinoma: Analysis of the pattern and risk factors,” Eur. J. Radiol.,vol. 59, no. 3, pp. 432–441, 2006.

[3] G. D. Dodd, M. S. Frank, S. Chopra, and K. N. Chintapalli, “Radiofre-quency thermal ablation: Computer analysis of the size of the thermalinjury created by overlapping ablations,” Amer. J. Roentgenol., vol. 177,no. 4, pp. 777–782, 2001.

[4] S. P. DiMaio, and S. E. Salcudean, “Needle steering and motion planningin soft tissues,” IEEE Trans. Biomed. Eng., vol. 52, no. 6, pp. 965–974,Jun. 2005.

[5] R. J. Webster III, J. S. Kim, N. J. Cowan, A. M. Okamura, andG. S. Chirikjian, “Nonholonomic modeling of needle steering,” Int. J.Robot. Res., vol. 25, no. 5, pp. 509–526, 2006.

[6] D. Glozman, and M. Shoham, “Image-guided robotic flexible needle steer-ing,” IEEE Trans. Robot., vol. 23, no. 3, pp. 459–467, Jun. 2007.

[7] V. G. Mallapragada, N. Sarkar, and T. K. Podder, “Robot-assisted real-time tumor manipulation for breast biopsy,” IEEE Trans. Robot., vol. 25,no. 2, pp. 316–324, Apr. 2009.

[8] R. J. Webster III, J. M. Romano, and N. J. Cowan, “Mechanics ofprecurved-tube continuum robots,” IEEE Trans. Robot., vol. 25, no. 1,pp. 67–78, Feb. 2009.

[9] N. A. Wood, K. Shahrour, M. C. Ost, and C. N. Riviere, “Needle steeringsystem using duty-cycled rotation for percutaneous kidney access,” inProc. Int. Conf. IEEE Eng. Med. Biol. Soc., 2010, pp. 5432–5435.

[10] A. Majewicz, S. P. Marra, M. G. van Vledder, M. Lin, M. A. Choti,D. Y. Song, and A. M. Okamura, “Behavior of tip-steerable needles inex vivo and in vivo tissue,” IEEE Trans. Biomed. Eng., vol. 59, no. 10,pp. 2705–2715, Oct. 2012.

[11] Z. Neubach, and M. Shoham, “Ultrasound-guided robot for flexible nee-dle steering,” IEEE Trans. Biomed. Eng., vol. 57, no. 4, pp. 799–805,Apr. 2010.

[12] S. Y. Ko, and F. Rodriguez y Baena, “Trajectory following for a flexibleprobe with state/input constraints: An approach based on model predictivecontrol,” Robot. Auton. Syst., vol. 60, pp. 509–521, 2012.

[13] K. B. Reed, A. Majewicz, V. Kallem, R. Alterovitz, K. Goldberg,N. J. Cowan, and A. M. Okamura, “Robot-assisted needle steer-ing,” IEEE Robot. Autom. Mag., vol. 18, no. 4, pp. 33–46, Dec.2011.

[14] R. Alterovitz, M. Branicky, and K. Goldberg, “Motion planning underuncertainty for image-guided medical needle steering,” Int. J. Robot. Res.,vol. 27, no. 11–12, pp. 1361–1374, 2008.

[15] V. Kallem, and N. J. Cowan, “Image guidance of flexible tip-steerableneedles,” IEEE Trans. Robot., vol. 25, no. 1, pp. 191–196, Feb.2009.

[16] K. B. Reed, A. M. Okamura, and N. J. Cowan, “Modeling and controlof needles with torsional friction,” IEEE Trans. Biomed. Eng., vol. 56,no. 12, pp. 2905–2916, Dec. 2009.

[17] N. A. Wood, C. A. Lehocky, and C. N. Riviere, “Algorithm for three-dimensional control of needle steering via duty-cycled rotation,” in Proc.IEEE Int. Conf. Mechatronics, 2013, pp. 237–241.

[18] M. C. Bernardes, B. V. Adorno, P. Poignet, and G. A. Borges, “Robot-assisted automatic insertion of steerable needles with closed-loop imagingfeedback and intraoperative trajectory replanning,” Mechatronics, vol. 23,pp. 630–645, 2013.

[19] D. C. Rucker, J. Das, H. B. Gilbert, P. J. Swaney, M. I. Miga, N. Sarkar, andR. J. Webster, “Sliding mode control of steerable seedles,” IEEE Trans.Robot., vol. 29, no. 5, pp. 1289–1299, Oct. 2013.

[20] M. Abayazid, G. J. Vrooijink, S. Patil, R. Alterovitz, and S. Misra, “Exper-imental evaluation of ultrasound-guided 3D needle steering in biologicaltissue,” Int. J. Comput. Assist. Radiol. Surg., to be published.

[21] T. R. Wedlick, and A. M. Okamura, “Characterization of robotic nee-dle insertion and rotation in artificial and ex vivo tissues,” in Proc.IEEE RAS EMBS Int. Conf. Biomed. Robot. Biomechatronics, 2012,pp. 62–68.

[22] S. Cheung and R. Rohling, “Enhancement of needle visibility inultrasound-guided percutaneous procedures,” Ultrasound Med. Biol.,vol. 30, no. 5, pp. 617–624, 2004.

[23] M. Ding, H. N. Cardinal, and A. Fenster, “Automatic needle segmentationin 3D ultrasound images using two orthogonal 2D image projections,”Med. Phys., vol. 30, no. 2, pp. 222–234, 2003.

[24] M. Ding and A. Fenster, “A real-time biopsy needle segmentation tech-nique using Hough transform,” Med. Phys., vol. 30, no. 8, pp. 2222–2233,2003.

[25] H. Zhou, W. Qiu, M. Ding, and Z. Songgen, “Automatic needle segmen-tation in 3D ultrasound images using 3D improved Hough transform,”in Proc. SPIE Med. Imag.: Image-Guided Procedures Modeling, 2008,vol. 6918, pp. 691 821–1–691 821–9.

[26] P. M. Novotny, J. A. Stoll, N. V. Vasilyev, P. J. del Nido, P. E. Dupont,T. E. Zickler, and R. D. Howe, “GPU based real-time instrument track-ing with three-dimensional ultrasound,” Med. Im. Anal., vol. 11, no. 5,pp. 458–464, 2007.

[27] M. Barva, M. Uhercik, J. M. Mari, J. Kybic, J. R. Duhamel, H. Liebgott,V. Hlavac, and C. Cachard, “Parallel integral projection transform forstraight electrode localization in 3-D ultrasound images,” IEEE Trans.Ultrason. Ferroelectr. Freq. Control, vol. 55, no. 7, pp. 1559–1569, Jul.2008.

[28] S. H. Okazawa, R. Ebrahimi, J. Chuang, R. N. Rohling, andS. E. Salcudean, “Methods for segmenting curved needles in ultrasoundimages,” Med. Im. Anal., vol. 10, no. 3, pp. 330–342, 2006.

[29] M. Aboofazeli, P. Abolmaesumi, P. Mousavi, and G. Fichtinger, “A newscheme for curved needle segmentation in three-dimensional ultrasoundimages,” in Proc. IEEE Int. Symp. Biomed. Imag.: Nano Macro, 2009,pp. 1067–1070.

[30] H. R. S. Neshat, and R. V. Patel, “Real-time parametric curvedneedle segmentation in 3D ultrasound images,” in Proc. IEEERAS EMBS Int. Conf. Biomed. Robot. Biomechatronics, 2008,pp. 670–675.

[31] M. Uhercık, J. Kybic, H. Liebgott, and C. Cachard, “Model fitting usingRANSAC for surgical tool localization in 3D ultrasound images,” IEEETrans. Biomed. Eng., vol. 57, no. 8, pp. 1907–1916, Aug. 2010.

[32] J. Holen, R. C. Waag, and R. Gramiak, “Representations of rapidly oscil-lating structures on the Doppler display,” Ultrasound Med. Biol., vol. 11,no. 2, pp. 267–272, 1985.

[33] G. Armstrong, L. Cardon, D. Vlkomerson, D. Lipson, J. Wong,L. L. Rodriguez, J. D. Thomas, and B. P. Griffin, “Localization of nee-dle tip with color Doppler during pericardiocentesis: In vitro validationand initial clinical application,” J. Amer. Soc. Echocardiography, vol. 14,pp. 29–37, 2001.

[34] R. Feld, L. Needleman, and B. Goldberg, “Use of a needle-vibratingdevice and color Doppler imaging for sonographically guided in-vasive procedures,” Amer. J. Roentgenol., vol. 168, pp. 255–256,1997.

[35] U. M. Hamper, B. L. Savader, and S. Sheth, “Improved needle-tip visu-alization by color Doppler sonography,” Amer. J. Roentgenol., vol. 156,no. 2, pp. 401–402, 1991.

[36] A. Harmat, R. N. Rohling, and S. E. Salcudean, “Needle tip localizationusing stylet vibration,” Ultrasound Med. Biol., vol. 32, no. 9, pp. 1339–1348, 2006.

[37] M. P. Fronheiser, S. F. Idriss, P. D. Wolf, and S. W. Smith, “Vibratinginterventional device detection using real-time 3-D color Doppler,” IEEETrans. Ultrason. Ferroelectr. Freq. Control, vol. 55, no. 6, pp. 1355–1362,Jun. 2008.

[38] K. E. Reddy, E. D. Light, D. J. Rivera, J. A. Kisslo, and S. W. Smith, “ColorDoppler imaging of cardiac catheters using vibrating motors,” UltrasonicImag., vol. 30, pp. 247–250, 2008.


[39] S. A. McAleavey, D. J. Rubens, and K. J. Parker, “Doppler ultrasoundimaging of magnetically vibrated brachytherapy seeds,” IEEE Trans.Biomed. Eng., vol. 50, no. 2, pp. 252–255, Feb. 2003.

[40] A. J. Rogers, E. D. Light, and S. W. Smith, “3-D ultrasound guidance ofautonomous robot for location of ferrous shrapnel,” IEEE Trans. Ultrason.Ferroelectr. Freq. Control, vol. 56, no. 7, pp. 1301–1303, Jul. 2009.

[41] T. K. Adebar and A. M. Okamura, “3D segmentation of curved needlesusing Doppler ultrasound and vibration,” in Proc. Int. Conf. Inf. Process.Comput.-Assisted Interventions, 2013, vol. 7915, pp. 61–70.

[42] H. Tchelepi and P. W. Ralls, “Color comet-tail artifact: Clinical applica-tions,” Amer. J. Roentgenol., vol. 192, no. 1, pp. 11–18, 2009.

[43] R. J. Webster III, J. Memisevic, and A. M. Okamura, “Design considera-tions for robotic needle steering,” in Proc. Int. Conf. Robot. Autom., 2005,pp. 3588–3594.

[44] T. Yoo, M. J. Ackerman, W. E. Lorensen, W. Schroeder, V. Chalana,S. Aylward, D. Metaxas, and R. Whitaker, “Engineering and algorithmdesign for an image processing API: A technical report on—The insighttoolkit,” in Proc. Med. Meets Virtual Reality, 2002, pp. 586–592.

[45] L. Crocetti, R. Lencioni, S. DeBeni, T. C. See, C. D. Pina, andC. Bartolozzi, “Targeting liver lesions for radiofrequency ablation: Anexperimental feasibility study using a CT-US fusion imaging system,”Invest. Radiol., vol. 43, no. 1, pp. 33–39, 2008.

[46] L. Maier-Hein, A. Tekbas, A. Seitel, F. Pianka, S. A. Muller, S. Satzl,S. Schawo, B. Radeleff, R. Tetzlaff, A. M. Franz, B. P. Muller-Stich,I. Wolf, H. U. Kauczor, B. M. Schmied, and H. P. Meinzer, “In vivo ac-curacy assessment of a needle-based navigation system for CT-guidedradiofrequency ablation of the liver,” Med. Phys., vol. 35, no. 12,pp. 5385–5396, 2008.

[47] T. Schubert, A. L. Jabob, M. Pansini, D. Liu, A. Gutzeit, and S. Kos, “CT-guided interventions using a free-hand, optical tracking system: Initialclinical experience,” Cardiovasc. Inter. Rad., vol. 36, pp. 1055–1062,2013.

[48] Y. S. Kim, W. J. Lee, H. Rhim, H. K. Lim, D. Choi, and J. Y. Lee,“The minimal ablative margin of radiofrequency ablation of hepatocellularcarcinoma (> 2 and < 5 cm) needed to prevent local tumor progression:3D quantitative assessment using CT image fusion,” Amer. J. Roentgenol.,vol. 195, no. 3, pp. 758–765, 2010.

[49] J. A. Engh, D. S. Minhas, D. Kondziolka, and C. N. Riviere, “Percutaneousintracerebral navigation by duty-cycled spinning of flexible bevel-tippedneedles,” Neurosurgery, vol. 67, no. 4, pp. 1117–1123, 2010.

[50] S. Patil, J. Burgner, R. J. Webster III, and R. Alterovitz, “Needle steeringin 3-D via rapid replanning,” IEEE Trans. Robot., to be published.

[51] P. J. Swaney, J. Burgner, H. B. Gilbert, and R. J. Webster III, “A flexture-based steerable needle: High curvature with reduced tissue damage,” IEEETrans. Biomed. Eng., vol. 60, no. 4, pp. 906–909, Apr. 2013.

Troy K. Adebar received the B.A.Sc. degree inmechanical engineering in 2009 and the M.A.Sc.degree in electrical and computer engineering in2011, both from The University of British Columbia,Vancouver, BC, Canada. He is currently working to-ward the Ph.D. degree in mechanical engineering atStanford University, Stanford, CA, USA.

His research interests include robotics, image-guided intervention, and medical devices.

Ashley E. Fletcher received the B.A.Sc. degree inmechanical engineering in 2012 from MassachusettsInstitute of Technology, Cambridge, MA, USA.She is currently working toward the M.S. degreein mechanical engineering at Stanford University,Stanford, CA, USA.

Allison M. Okamura (F’11) received the B.S. de-gree from the University of California, Berkeley,CA, USA, in 1994, and the M.S. and Ph.D. degreesfrom Stanford University, Stanford, CA, in 1996 and2000, respectively, all in mechanical engineering.

She is currently an Associate Professor of mechan-ical engineering at Stanford University, Stanford, CA,USA, where she is also the Robert Bosch FacultyScholar. Her research interests include haptics, tele-operation, medical robotics, virtual environments andsimulation, neuromechanics and rehabilitation, pros-

thetics, and engineering education.Dr. Okamura received the 2004 National Science Foundation CAREER

Award, the 2005 IEEE Robotics and Automation Society Early Academic Ca-reer Award, and the 2009 IEEE Technical Committee on Haptics Early CareerAward. She is an Associate Editor of the IEEE TRANSACTIONS ON HAPTICS.

3-D Ultrasound-Guided Robotic Needle Steering in Biological...

Documents

Transcript of 3-D Ultrasound-Guided Robotic Needle Steering in Biological...