A Novel Robot-Assisted Endovascular Catheterization System ...guolab.org/Papers/2019/song...

IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 3, JUNE 2019 685

A Novel Robot-Assisted EndovascularCatheterization System With

Haptic Force FeedbackShuxiang Guo, Senior Member, IEEE, Yu Song , Xuanchun Yin, Linshuai Zhang,

Takashi Tamiya, Hideyuki Hirata, and Hidenori Ishihara

Abstract—The robot-assisted endovascular catheterization sys-tem (RAECS) has the potential to address some of the proceduralchallenges and separate interventionalists from the X-ray radia-tion during the endovascular surgery. However, the employmentof robotic systems is partly changing the natural gestures and be-havior of medical professionals. This paper presents a RAECS thataugments surgeon’s motions using the conventional catheter as wellas generates the haptic force feedback to ensure surgery safety.The magnetorheological fluids based haptic interfaceexperimentis proposed to measure the actions of the operator and providethe haptic force. The slave catheter manipulator is presented toactuate the patient catheter in two degrees of freedom (radial andaxial direction). In addition, a force model is provided to charac-terize the kinematics of the catheter intervention. Afterward, the“pseudocollision” and “real collision” are utilized to descript thecatheter tip–vessel interaction. The in vitro results are presented,which demonstrate that catheter-based haptic force feedback hasa benefit for improving catheterization skills as well as reducingthe cognitive workload of operators.

Index Terms—Haptic force feedback, magnetorheological (MR)fluids, master-slave, robot-assisted endovascular catheterizationsystem (RAECS).

Manuscript received July 16, 2018; accepted January 17, 2019. Date of pub-lication March 12, 2019; date of current version May 31, 2019. This work wassupported in part by the National High-Tech Research and Development Pro-gram (863 Program) of China under Grant 2015AA043202, and in part by SPSKAKENHI under Grant 15K2120. This paper was recommended for publica-tion by Associate Editor K. Xu and Editor P. Dupont upon evaluation of thereviewers’ comments. (Corresponding author: Yu Song.)

S. Guo is with the Key Laboratory of Convergence Medical Engineering Sys-tem and Healthcare Technology, Ministry of Industry and Information Technol-ogy, Beijing Institute of Technology, Beijing 100081, China, and also with theIntelligent of Mechanical System Engineering Department, Kagawa University,Takamatsu 761-0396, Japan (e-mail:,[email protected]).

Y. Song is with the Tianjin Key Laboratory for Control Theory & Applicationsin Complicated Systems, Tianjin University of Technology, Tianjin 300384,China (e-mail:,[email protected]).

X. Yin is with the College of Engineering, South China Agricultural Univer-sity, Guangzhou 510642, China (e-mail:,[email protected]).

L. Zhang is with the Graduate School of Engineering, Kagawa University,Takamatsu 761-0396, Japan (e-mail:,[email protected]).

T. Tamiya is with the Department of Neurological Surgery Faculty ofMedicine, Kagawa University, Takamatsu 761-0396, Japan (e-mail:,[email protected]).

H. Hirata and H. Ishihara are with the Intelligent of Mechanical Sys-tem Engineering Department, Kagawa University, Takamatsu 761-0396, Japan(e-mail:,[email protected]; [email protected]).

Color versions of one or more of the figures in this paper are available onlineat http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TRO.2019.2896763

I. INTRODUCTION

SHORT recovery time, a small incision to the healthy issue,good surgical outcomes, and little postoperative pain have

facilitated the adoption of endovascular surgical techniques [1].But for interventionalists, long fluoroscopy times and X-ray ra-diation exposure to them lead to problems, such as cancers andcataracts [2]. Even though lead aprons are worn, the face andhands are still exposed. The heavy protection suits with longhours in the operating can lead to neck and back pain and in-jury [3]. This potential shortcoming has been the motivationto develop a robot-assisted endovascular catheterization system(RAECS). Unlike the conventional bedside catheterization tech-nique, the RAECS allows the interventionalist to offer axial andradial action by the master robot placed in a remote location.The control commands are processed by the computer console,to control the catheter manipulator to insert, retract, or rotate thepatient catheter.

Robot-assisted surgeries have been adopted in the medicalfield, especially in minimally invasive surgery. The develop-ment of a teleoperated surgical robot is motivated by the de-sire to enhance the effectiveness of the surgical procedure andprovide a comfortable operating environment for the surgeon[4]. To ensure surgery safety, an ideal teleoperated surgery sce-nario is viewed as a physical extension of the human body.Hence, a high level of transparency is essential to an operator tomake the correct decision in a human-centered teleoperation sys-tem. Surgical skills are determined by information acquisition:medical knowledge accumulation (atlases of human anatomy),physical experience (kinesthetic sensation and tactile sensation),and preoperation and intraoperation patient-specific data (vitalsigns and images) [5]. For example, the understanding of tissuecharacteristics is a determining factor for a successful teleop-erated robot-assisted surgery [6]. Okamura et al. pointed outthat lacking natural haptic sensation was the main limitationin telesurgery because the surgeon was physically separatedfar from the patient [7]. Nowadays, the haptic technology inteleoperated surgery is a promising research area. Some com-mercial haptic devices have been developed, such as Sigma 7,Omega, and Premium 3.0 master hand-controller, which havebeen applied in a robot-assisted surgery system [8]. Pacchierottiet al. integrated haptic sensation (kinesthetic and vibratory in-formation) in the master interface for the teleoperated needle

1552-3098 © 2019 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

https://orcid.org/0000-0002-9295-7795

mailto:[email protected]








686 IEEE TRANSACTIONS ON ROBOTICS, VOL. 35, NO. 3, JUNE 2019

insertion [9]. The target of these researchers was to developa high quality of transparency and enhance the performanceof the robot-assisted surgery system by introducing the haptictechnology [10].

In recent years, some research groups have developedRAECSs, which were combined with the haptic feedback, suchas [11]–[14]. However, the master devices in these systems areall motor-based haptic devices. Therefore, some inherent fac-tors of motor-based devices, such as the damping, inertia, andfriction, will significantly reduce the transparency of the system[15]. Additionally, surgeons strongly rely on their sense of touchand their intuitive skills during an endovascular procedure. Theemployment of these haptic devices is potentially changing thenatural gestures and behavior of medical professionals.

The main contribution of this paper is recreating catheter-based haptic force sensation by the magnetorheological (MR)fluids based master haptic interface, which can improve thequality of transparency between the slave side and master side.The MR fluids actuated haptic interface is as the master robotto provide haptic sensation. The control command of cathetermanipulation in the master haptic interface can be replicatedby the slave catheter manipulator. The catheter tip force sensorhas been developed to measure the interaction force betweenthe catheter tip and blood vessel. The endovascular catheteriza-tion dynamic model and “pseudocollision” and “real collision”discrimination theory are established. Then, the haptic forcefeedback is designed to assist the operator to obtain an optimumstrategy of catheterization in the blood vessel phantom. In ad-dition, the system design and proposed haptic force feedbackcontrol strategy are validated via in vitro experimental results.

The remainder of this paper is organized as follows. TheRAECS is described in Section II. Section III is devotedto establishing a dynamic model of catheter intervention. InSection IV, the haptic force feedback strategy is described indetail. The experiments have been carried out to evaluate the per-formance of RAECS with haptic force feedback in Section V.Finally, discussion and conclusion are presented in Sections VIand VII, respectively.

II. SYSTEM DESIGN

The proposed RAECS comprises three main parts. The mas-ter haptic interface (to be placed at a local site) measures theaxial and radial motions of an input catheter, which is oper-ated by the interventionalist. Meanwhile, the haptic feedbackcan be provided to the interventionalist during the operation.The catheter manipulator replicates the interventionalist’s op-eration motions, which are measured by the sensor part of themaster haptic interface. Using image guidance, the interven-tionalist tracks the catheter tip position and remotely navigatesthe catheter to the desired target. The computer console is usedto relay information between the master device and the slavemanipulator.

A. Master Haptic Interface

The design of the master robot should contain two capabili-ties, measuring the axial and radial motions of the input catheter(7 F catheter) and applying the force feedback with the haptic

Fig. 1. Master haptic interface.

Fig. 2. (a) Working principle of catheter motion measurement. (b) Schematicdiagram of MR fluids container.

interface. Fig. 1 shows the proposed master haptic interface.The laser-based navigation sensor was utilized to measure theaxial motion and radial motion of the input catheter, as shownin Fig. 2(a). The vertical-cavity surface- emitting laser emits thelaser light to the surface of the catheter, and then the optical sen-sor will receive the image of this detected surface. Hence, whenthe input catheter is pulled, pushed, or rotated, the changes oftwo adjacent surface images, both in axial and radial directions,can be detected.

The MR fluids container is a cylinder-like structure, its innerdiameter is 10 mm and length is 100 mm. The catheter goesthrough the MR fluids, and it is coaxial with the container, asshown in Fig. 2(b). In order to prevent the leakage and decreasethe frictional resistance, the four small permanent magnets arefixed at both ends of the container. Four sponges are used toabsorb the leakage fluids [16].

The haptic force is generated by the MR fluids based actuatedhaptic interface. Actuated haptic interface of MR fluids, whichare a kind of smart material, was developed in the past decade[17], [18]. Haptic interfaces based on MR fluids have severaladvantages over traditional ones, such as fast response time, lowpower consumption, and intrinsic passivity [19]. The viscosityof MR fluids can be adjusted by an external magnetic field.When the operator navigates the input catheter through the hap-tic interface under the magnetic field, the chain-like structure ofMR fluids particles will be distorted and the passive force canbe produced. In particular, the operator’s gestures match withthe traditional catheter interventional practice.

The generated resistance forces of the input catheter inser-tion are mainly influenced by the shear stress of the MR fluids,

GUO et al.: NOVEL ROBOT-ASSISTED ENDOVASCULAR CATHETERIZATION SYSTEM WITH HAPTIC FORCE FEEDBACK 687

Fig. 3. Catheter manipulator, the two degrees of freedom robotic device pulls,pushes, and steers the patient catheter.

which are proportional to external electromagnetic powers [20].In addition, when the magnetic field is applied, the MR fluidswill play a non-Newtonian behavior; hence, the insertion speedof the input catheter will have an effect on perceived resistanceforce of operators [21]. The resistance force is related to the con-trollable yield stress, and the detailed description is presented in[22]. Two Hall sensors are embedded in an MR fluids containerholder to obtain the magnetic field intensity. The maximumgenerated resistance force is over 0.5 N. This paper continuesresearch based on the designed master haptic interface, the de-sign details and the performance evaluations were described byour publications [16], [20], [22], [23].

B. Catheter Manipulator

The patient catheter is navigated via the blood vessel by thecatheter manipulator as commanded by the surgeon using themaster haptic interface. The catheter manipulator is shown inFig. 3. The grasper A is controlled by a dc gear motor to hold thecatheter for insertion, retraction, and rotation motions. When theplatform moves to the front end, the grasper B (controlled by dcgear motor) will be clamped to keep the position of the catheter,and then grasper A will release the catheter. Then, the platformwill be retreated for the next trail. To prevent slipping of thecatheter, the generated axial gripping force of two graspers is4 N, and the radial gripping torque of grasper A is 6 N·m, whichmeets the maximum value of force and torque applied by in-terventionalists during conventional endovascular surgery [24].Two stepping motors (ASM46AA, Oriental Motor, Japan) areutilized to actuate the platform in radial and axial directions. Nostep loss can be achieved by a high-performance microsteppingdriver (EN50178, Oriental Motor, Japan), even abrupt load vari-ation. The distal force of the catheter is measured by the loadcell (TU-UJ, TEAC, Japan), and the value will be displayed onthe console screen [22].

C. Console

The control of the master site and slave site is achievedthrough a computer console via RS-232 serial communication.Control software is implemented using C++. The schematic

Fig. 4. System block diagram of the RAECS.

diagram of the RAECS is presented in Fig. 4. The motiondata of the master haptic interface are transmitted to the com-puter console by the ad board (AD 16-16U (PCI), CONTEC),and the motion command is sent to the motor actuators ofthe slave catheter manipulator via the motion control board(SMC-4DF-PCI, CONTEC).

III. DYNAMICS OF CATHETER INTERVENTION

In this section, the model of catheter interventional is pre-sented, and then the catheter tip–vessel collision discriminationtheory is carried out.

A. Model of Catheter Intervention

In the catheterization procedure, two different kinds of forcecan be perceived by the interventionalist. One is the frictionforce which is required to advance the catheter through theintroducer sheath. Another is the interaction force between thecatheter and the vascular system. This interaction force can bedivided into three classifications: no contact and no collision,contact, and collision. In the first situation, the operator onlyfeels the viscous drag force which is caused by the viscosity ofthe blood. This kind of force exists in the whole interventionalprocedure. In the second situation, when the catheter is insertedinto tortuous parts of the vascular system, friction force will begenerated due to catheter surface contacts with the vessel wall.In the third situation, catheter tip will be contacted with the innerwall of the vascular system, the surgeon must vary the operationgesture (retreat or rotate the catheter) to change the direction ofthe catheter tip. Compared to the analysis in [25], these forcescan be expressed as follows:

fvdf = c · v (1)

fcff = μ · fN (2)

fcf = E · S · δ (3)

where fvdf , fcff , and fcf are viscous drag force, contact frictionforce, and collision force, respectively. c, μ, and E are the vis-cosity coefficient of the blood, the dynamic friction coefficientgenerated between catheter surface and blood vessel wall, andYoung’s modulus of blood vessel, respectively. v, fN , S, and δare the velocity of catheter intervention, the normal force actedon the catheter, the cross-sectional area of the catheter tip, andthe magnitude of the deformation of blood vessel, respectively.The total force of catheter intervention can be expressed using


Fig. 5. Situations of “pseudocollision” and “real collision.”

the following equations:

Fdh = fis + fvdf + sgnfcff + sgnfcf (4)

sgn =

{1, (contact or collision)

0, (no contact or no collision)(5)

where Fdh is the direct haptic sensation of the surgeon ac-quired during endovascular surgery. fis is the force to overcomefriction between the catheter and an introducer sheath, the min-imum value is 0.29 ± 0.06 N [26]. The value of sgn is de-termined by the situation of catheter intervention. The viscousdrag force and contact friction force are difficult to measure andestimate in real time, due to the dynamic change of blood ves-sel and the unfixed contact points of the catheter body with theblood vessel wall. The collision force between the catheter tipand vessel wall is proportional to the elastic deformation of theblood vessel wall.

B. “Real Collision” and “Pseudocollision”

In practice, the success of endovascular surgery is depen-dent on the discrimination of collision by a skilled surgeon indirect operation. The interaction between the catheter tip andblood vessel wall exists in the catheter interventional proce-dure. It needs to point out that it is impossible without contactbetween the catheter tip and blood vessel during catheter op-eration practice [23]. The contact forces are ones that requirea collision, such as a catheter tip hitting the inner blood vesselwall. Therefore, only when the contact force exceeds the safetythreshold value (deformation of blood vessel δ exceeds elasticlimit δmax ), it can be viewed as “real collision.” If the defor-mation of blood vessel δ does not exceed elastic limit δmax , itis viewed as “pseudocollision.” The situations of “pseudocolli-sion” and “real collision” are shown in Fig. 5. The deformationδ is proportional to the impact force fcf between the cathetertip and the inner vessel wall. Therefore, collision discrimination

Fig. 6. Catheter intervention process.

can be given by

fcf =

{fcf < Fmax ,

′pseudo collision′

fcf ≥ Fmax ,′real collision′ (6)

where Fmax is the threshold safety value. It is noted that hap-tic sensation has a close relationship with interaction situa-tion between the catheter and vascular system during catheterinterventional. Obviously, the haptic sensation is a kind ofnoncontinuous force.

IV. HAPTIC FORCE FEEDBACK

A. Design of Haptic Force Feedback

In this design, the interventionalist is guided by haptic feed-back during the catheter tip–vessel contact. The recreated guid-ing force should be proportional to the elastic deformation ofthe vascular wall and can be perceived by the surgeon to avoidthe penetration of vascular wall. Substituting (3) into (6) gives

δ ≤ Fmax

E= δmax (7)

where δmax is the elastic limit; in order to ensure surgery safety,it should keep δmax in a safe deformation range of vascular walltissue. To calibrate δmax , Young’s modulus E and Fmax shouldbe known. The range of E can be defined by the ages and sexof the patients and different parts of the blood vessel.

Blood vessel deformation information can hardly be obtainedby the intraoperative image processing program by 2-D fluo-roscopy images, but the contact force can be obtained by thecatheter tip force sensor. During catheter intervention, when thecatheter tip collides with the inner wall of the blood vessel, de-formation of the blood vessel will happen. The deformation δis proportional to the impact force between the catheter tip andthe inner vessel wall. In Fig. 6, the dashed line shows the centerline of the blood vessel. In an ideal situation, the trajectory ofcatheter insertion is along the center line to the disease target.However, it is impossible to achieve that goal during endovas-cular procedure, even by an experienced interventionalist. The“pseudocollision” always happens but hardly causes a medicalaccident. The red line shows the deformation of the blood vesselafter “pseudocollision” or “real collision.”

Here, the different values of collision forces between thecatheter tip and blood vessel wall will be selected to describe


Fig. 7. (a) Force calibration setup. (b) Fibre optical sensor mounted inside the5 F catheter.

the deformation of the blood vessel as follows:

Fhg =

{kP FPseudo, 0 < δ < δmax

kRFReal, δ ≥ δmax(8)

where Fhg is the haptic guidance (HG) perceived by the oper-ator. FPseudo and FReal are utilized to describe the collisionforces of “real collision” and “pseudocollision,” respectively.The kP and kR are coefficients for FPseudo and FReal , respec-tively. The thoracic aorta blood vessel of 50–60 years’ peoplewill be damaged by the direct force 0.12 N [26]. In this design,the collision force between the catheter tip and blood vesselexceeds 0.12 N, which means the “real collision” happened.

For a designed haptic force feedback strategy, the differencesof the generated resistance forces should be noticed by the oper-ators. Human HG perception often obeys Weber’s law. Weber’slaw is given by

Δ∅∅0

= k (9)

where Δ∅ is the just noticeable difference (JND) [28]. It de-scribes the minimum difference between two stimuli, which canbe reliably perceived by operators. ∅0 is the initial stimulus andthe constant k is Weber fraction.

B. Catheter Tip Force Calibration

In order to obtain contact forces between the catheter tip andvascular wall, a fiber optical sensor (diameter 1.3 mm, FISOTechnologies Inc.) integrates with the catheter, as a tube-in-tubestructure is utilized to measure tip forces. The experiment isset up to investigate the output of the fiber optical sensor inaxial loading, as shown in Fig. 7(a). The catheter goes throughthe stopcock, which in the distal end is grasped without slip.The stopcock is fixed with the transducer rod of load cell by aclamper. The load cell is mounted on the platform of linear stagethat is actuated by a stepping motor. The catheter is navigated inthe horizontal direction at the speed of 10 mm/s. A soft silicone-based phantom is utilized as soft tissue to test the collision force.The sensitive area of fiber optical pressure sensor is fragile, sothe test collision force will not exceed 0.2 N. Hence, when themeasured collision force is 0.2 N, the platform will go back forthe next trail.

The result of stress against force in the axial loading is shownin Fig. 8, loading and unloading in five times. The experimental

Fig. 8. Stress against force in the axial loading.

Fig. 9. Schematic diagram of the control loop structure for the RAECS.

result shows a good linear relationship between the stress and themeasured force in axial direction. The sensor also reflects goodsensitivity in collision with the soft silicone-based phantom.The force working range is estimated from 0 to 0.2 N; thecorresponding pressure is from 13.40 to 23.39 lb/in2.

C. Control Architecture

The schematic diagram of the control loop structure for theRAECS has been shown in Fig. 9. The system goal is to avoid the“real collision” during the endovascular procedure. The contactforce between the catheter tip and blood vessel wall is obtainedby the designed catheter tip sensor. The controller is utilized todistinguish the different kinds of collision (no collision, “pseu-docollision,” and “real collision”). The force control model isutilized to describe the relationship between the generated hap-tic resistance force and applied magnetic field of master hapticinterface [19]. The intensity of the magnetic field is measuredby embedded hall sensors inside the MR fluids container. Theerror between the desired magnetic field and its measured valuesprovides a reference signal for a current source, which actuatesthe electromagnetic coils. The passive HG can be perceived bythe human operator during the operation. The surgeon’s actions


Fig. 10. Catheter is inside the rigid model of cerebral vascular.

are captured by the master haptic interface to guide the slavecatheter manipulator. In addition, the visual feedback (VF) canassist the surgeon in navigating the patient catheter.

V. PERFORMANCE EVALUATION

In this section, three experiments are carried out to eval-uate the performance of RAECS with haptic force feedback.Experiment I is used to test the tracking performance of themaster–slave system. Experiment II is utilized to find the JND ofresistance force during the input catheter insertion.Experiment III is used to verify the effectiveness of haptic forcefeedback strategy in vitro.

A. Performance Evaluation of the System

1) Experimental Method: In order to evaluate the accuracyof designed RAECS, the ten participants were recruited to ma-nipulate the input catheter through the master haptic interfaceto guide the catheter manipulator to actuate a 5 F catheter incerebral vascular phantom, as shown in Fig. 10. The VF wasprovided by a round view camera, which was mounted rightabove the phantom. The operators had no interventional expe-rience or experience of using RAECS. Inexperienced operatorswere selected because of their higher kinematics in catheter nav-igation than professionals [26]. Before the experiment, the par-ticipants were informed about the procedure and provided with15 min of training on this system. The 5 F catheter should benavigated through the rigid model of cerebral vascular, and thenthe catheter tip should be directed into the wide neck aneurysm,as shown in Fig. 10. Each participant repeated the procedure tentimes. The operator motions have filtered the frequencies be-yond the range of 5–20 Hz to remove hand tremor. The hollowencoder (UN-2000, MUTOH, Japan) connects with the catheternavigator to measure the rotation angle of catheter manipulator.Axial displacement of catheter manipulator is measured by thelaser sensor (LK 5000, Laser displacement sensor, KEYENCECorp, Japan).

2) Experimental Results: The accuracy of the catheter nav-igation in radial and axial directions is shown in Fig. 11. It wasfound that the mean errors between applied motion by master

Fig. 11. Results of catheter insertion experiment. The accuracy of catheternavigation by inexperienced subjects. (a) In the axial direction. (b) In the radialdirection. The data are the median, lower median, upper median, and range ofmeasured errors for ten trails per subject.

Fig. 12. Schematic of resistance force resolution experiment.

haptic interface and output motion of catheter manipulator inaxial and radial directions were 1.23 ± 0.07 mm (range: 0.5–2.1 mm) and 1.57 ± 0.07° (range: 0.9–2.1°), respectively.

B. Resistance Force Difference Experiment

For a conventional bedside catheterization technique, onlythe sudden change of resistance force of patient catheter ma-nipulation can be perceived by interventionalist. In addition, theinterventionalist can hardly distinguish the collision force fromother resistance forces during the catheterization procedure [23].Pang et al. indicated that a JND varied between100 and 200 mNfor pinching motions from finger and thumb by constant resis-tance forces [29], considering that the biggest resistance forcegenerated by the designed master haptic interface is about 0.5 N(when the applied magnetic flux density is 150 mT) [20]. Thespecific values of resistance force were selected as the differentlevels of haptic feedback to the operator, and the differencesbetween adjacent forces can be just noticeable by operators.

1) Experimental Method: The goal of this experiment is todetermine the JND of the resistance force that the operator’sfinger is able to detect while navigating a catheter through thedesigned master haptic interface. The schematic of resistanceforce resolution experiment is shown in Fig. 12. The appliedmagnetic field will be increased or decreased in fixed steps of10 mT. The reference force is about 0.4 N and the correspond-ing applied magnetic field is 110 mT. Ten subjects, according toendovascular experience and haptic knowledge, were separatedinto two groups: experienced (n = 4, with rich endovascular ex-periences and haptic device operation skills) and inexperienced


Fig. 13. Results of the force in different experiments by ten operators, and thereference force were selected at the 110 mT magnetic flux density.

(n = 6, no endovascular experiences, and haptic knowledge).All subjects were right-handed. The operators were instructedto concentrate on the force perception in their figures. Eachoperator was asked to perform the task ten times at each fixedstep. They were allowed to vary the test force as often as theylike, while they could always insert the catheter and check ifthe resistance force feels constant or not. The applied test forceis bigger or smaller than the reference force, which was cho-sen randomly in sequential trails. The difference between thereference magnetic field and test magnetic field at the start ofeach trial was randomly distributed between 10 and 110 mT.Before commencing the operation, all operators underwent ashort training session in order to familiarize themselves withthe use of the designed haptic interface.

2) Experimental Results: The results of the force in differentexperiments are shown in Fig. 13. When the applied magneticfield is smaller than the reference magnetic field, all operatorscan notice the changes of resistance force at 30 mT (resistanceforce is about 0.16 N). In the case of the magnetic field at 80 mT(resistance force is about 0.3 N), in more than 90% of trailsthe operators can feel the change between the test force andthe reference force. As can be seen, when the magnetic field is150 mT, the discrimination rate is 93%.

C. Evaluation of the Haptic Force Feedback Strategy in EVE

1) Experimental Method: The experimental setup, shown inFig. 14, consists of two parts. In the remote site, a silicone-basedEVE (EndoVascular Evaluator) model is used for this paper. Thecatheter manipulator actuates the patient catheter into the EVEmodel. In order to simulate 2-D fluoroscopy guidance duringthe experimental process, a round view camera is mounted rightabove the EVE model to get the visual feedback. The images ob-tained from the camera are processed using brightness, contrast,and color adjustments, so as to enhance the location image ofcatheter tip. In the local site, the image information is projectedon the computer console screen, and the haptic force feedback isprovided to interventionalist for operating the catheter throughthe haptic interface.

Fig. 14. Experimental setup of the RAECS for endovascular procedure.

In order to simulate the real operation environment, the en-dovascular surgery training simulation EVE was utilized toprovide a realistic immersive virtual reality environment. Inaddition, a kind of water–glycerin mixture fluid (50.6% by vol-ume) was used to create a real blood. This fluid has a dynamicviscosity of 6.9 cP and the density of 1.02 g/cm3 [30]. Theflow rate can be changed by the pulsed flow pump (Iwaki Bel-lows Pump, Japan). Moreover, a digital manometer is utilizedto measure the blood pressure in real time. The 5 F (diameter 2mm) catheter integrated with the optic fiber pressure sensor isnavigated by the catheter manipulator.

For evaluating our prototype system and control strategy, weconducted the experiment: the operators navigated the operationcatheter through the master haptic interface to guide a patientcatheter from the femoral artery to the common carotid in theEVE model. The same ten participants of Experiment B tookpart in this experiment. Therefore, it needs hand–eye coordi-nation when the operators manipulate the input catheter in themaster site. Experimental conditions have been divided into thefollowing two groups.

1) No haptic guidance (NHG): only the VF information isprovided to the volunteer operator.

2) HG: both the HG and VF are provided to operators.Depending on the previous force in different experiment,

three degrees of haptic force were provided to operators, differ-ent degrees corresponded to different ranges of collision force.When the collision force is below 0.08 N, the navigation ofcatheter is in the absence of a magnetic field; when the value ofcollision force is at the range of 0.08–0.1 N or 0.1–0.12 N, itmeans the two degrees of “pseudocollision” will occur and theapplied magnetic fields are 80 mT and 110 mT, respectively. Ifthe value of collision forces exceeds 0.12 N, it is recognized as a“real collision” and the applied magnetic field is 150 mT. Eachoperator is asked to perform the task ten times in each condi-tion. Each trial is considered an independent test. The subjectsunderwent a short training to familiarize themselves with theuse of the system before commencing the experiments.

2) Data Acquisition and Metrics: For the master and slavesides, the time, forces, and positions were recorded at 1 kHz.The recorded data serve to evaluate the task performance. Thetask performance is evaluated as follows.


Fig. 15. Mean values of task-completion times (a) and distances traveled of apatient catheter (b) by different levels of operators in HG or NHG.

1) Task-completion time, representing the time is required tocomplete the endovascular task.

2) Path length, representing the total path length of catheternavigation. The path length of the catheter is an absolutevalue; retreat and insertion movements are all calculated.

3) Contact force, representing the collision force betweenthe catheter tip and vessel wall.

3) Experimental Results: Fig. 15 illustrates the mean val-ues of task-completion times and distances traveled of a patientcatheter for each operator under two different conditions. FromFig. 15(a), the experienced operators needed less time in cathetermanipulation than inexperienced in the same condition. For thetwo conditions, we found a significant difference in the meancompletion times of two different levels, which indicated that thevalue of the completion time was affected by haptic guidance.The mean values were decreased to 5.6% and 4.0% for inexperi-enced and experienced operators, respectively, when they haveapplied the haptic guidance. Fig. 15(b) shows the mean valuesof distances of catheter navigation during the two conditions

TABLE IMEANS OF THE MAXIMUM CONTACT FORCE VALUES AND THE MEANS OF THE

CONTACT FORCE VALUES FOR EXPERT AND NOVICE OPERATORS

BY HG OR NHG

TABLE IIMEANS OF THE TIMES WHEN THE CONTACT FORCES BETWEEN THE CATHETER

TIP AND BLOOD VESSEL ARE GREATER THAN OR EQUAL TO

0.08 N, 0.1 N, AND 0.12 N

by the different levels of operators. The distance of cathetermovement includes the retreat and advance displacement. Asobserved, the experienced operators needed less operating dis-tance to complete the task. Moreover, there was a significantdifference in the mean values of distances by HG or NHG. Themean values were reduced by 7.8% and 6.5% for inexperiencedand experienced operator, respectively, when haptic guidance isapplied.

Table I illustrates the means of the maximum contact forcevalues and the means of the contact force values in the axialdirection. The lower contact forces were exerted by experi-enced operators, rather than novices, for the task. It also showsthat the lower contact forces were applied by experienced andinexperienced operators under HG rather than NHG.

Table II illustrates the mean times when the contact forces aregreater than or equal to 0.08 N, 0.1 N, and 0.12 N. Experiencedoperators spend less time on “pseudocollision” and “real colli-sion” than inexperienced in the same condition. It also showsthat both the experienced and inexperienced operators makefewer collisions under HG and NHG.

Fig. 16 shows examples of the contact force between thecatheter tip and blood vessel for experienced versus novice op-erators (subject 1 versus subject 7) under the HG or NHG. Inthe case of haptic guidance, both the task performance time andcontact force were reduced than those in NHG by inexperienceand experienced operator, respectively. In two conditions, boththe subject 1 and subject 7 generated small contact forces at thepart of the thoracic aorta and descending aorta. While the vesseltortuosity and angulation cause difficulties in catheter naviga-tion, the large forces occurred at procedural of the aortic archand right common carotid artery.

Figs. 17 and 18 illustrate the relationships between contactforces curves and applied magnetic field curves in proceduralof the aortic arch and right common carotid artery, by subject 1and subject 7, respectively. It was seen that the applied magnetic


Fig. 16. Examples of contact force signals for experienced versus inexperi-enced operators in HG or NHG. The three procedural phased are depicted indifferent colors.

Fig. 17. Examples of the contact force versus applied magnetic field for theinexperienced operator (subject 1) in haptic guidance. (a) Aortic arch. (b) Rightcommon carotid artery.

Fig. 18. Examples of the contact force versus applied magnetic field for theexperienced operator (subject 7) in haptic guidance. (a) Aortic arch. (b) Rightcommon carotid artery.

field was quickly established when the contact force exceededthe setting values (“pseudocollision” and “real collision” hap-pened). Meanwhile, the passive haptic resistance force will beprovided to operators when they navigate the input catheter.

VI. DISCUSSION

In this paper, a novel RAECS has been presented for roboticendovascular surgery. The system enables interventionalists touse their conventional skills to remotely navigate the catheter,and it may have the potential to reduce the exposure time ofthe patient and interventionalists to X-ray radiation [32]. Theinteresting feature of the master haptic interface is that the in-put catheter has been utilized to provide the motion commandsand transfer the haptic force feedback to operators. Comparedwith other RAECSs [29], [33], the commercially available joy-sticks or haptic interface device, such as Omega, was as themaster robot, which removed the catheter from the interven-tionalists’ hands, and thus, they cannot apply their conventionalbedside catheterization technique skills during the procedure.Therefore, some researchers [34], [35] had addressed this by


developed catheter-based master devices, but those lacked hapticforce feedback.

The simplicity of an experiment was performed to evaluatethe accuracy of the master–slave system. The average accuracyof catheter navigation in the axial and radial directions wasbetter than 2 mm and 2°, respectively (see Fig. 11). The errorcould be attributed to two reasons: First, two graspers of cathetermanipulator need enough motion time to clamp and release thecatheter; also increasing the number of directional changes mayresult in decreasing motion accuracy. Second, there is a lag timebetween sensed and replicated motion. Although these issuesalways existed, the error can be partly offset in practice bythe operators, who will use fluoroscopic imaging as positionfeedback of the catheter tip.

Three force level thresholds were selected to represent inter-action situations between the catheter tip and blood vessel wall.These thresholds were used to alert the operators that the “pseu-docollision” or “real collision” happened. From Table I, themeans of the maximum contact force was decreased when theHG was provided to both experienced and inexperienced opera-tors. The experimental results indicate that HG has the ability toassist operators in decision-making to change the manipulationgestures to decrease the contact force, although the operatorswere reminded that the “real collision” (the magnetic field is 150mT) was the final caution. Unfortunately, during the experiment,the maximum contact force was not decreased below 0.12 Nwhen the haptic force was applied. Hence, for the safety ofsurgery, when the measured contact force exceeds the thresholdsafety value Fmax , all catheter movements of the catheter ma-nipulator should be stopped immediately. Moreover, the cathetermanipulator must be separated from the patient catheter freely,and the interventionalists could use their conventional skills togo on the surgery. For the further real application, the structureof the catheter manipulator should be further optimized.

The designed haptic force feedback scheme had substantiallyreduced the total path length of the catheter, shortened the pro-cedure time, (see Fig. 15) and decreased the duration times of“pseudocollision” and “real collision” (see Table II) comparedto NHG during the manipulation by both the inexperiencedand experienced operators. This means that the operation skillspotentially improved; such benefit of haptic force feedback intask implementation was reported in many other studies [9],[36], [37]. Moreover, the reduction in path length and proceduretime could potentially translate into reduced human cognitiveworkload during the operation. The reduction in times of “pseu-docollision” and “real collision” could potentially translate intoreduced vessel wall contact and therefore less risk of penetratingthe vessel.

It is important to note that the input and output of the MRfluids based device are nonlinearly related to hysteresis [17].From Figs. 17 and 18, when the “pseudocollision” or “real col-lision” disappeared, the magnetic field did not decrease sharply.In addition, if the time interval of collisions is short, the re-sponse of the magnetic field will be a little delayed. As shownin Fig. 17(a), the contact force was beyond 0.12 N, but themagnetic field density was less than 150 mT. This may have aninfluence on the accuracy and rapidity of the passive resistance

force generation. The hysteresis phenomenon within the hapticinterface is mainly associated with ferromagnetic componentsincluding iron cores and coils. The little or no hysteresis is ob-served in the B–H curve of MR fluids [19]. Transparency is oneof the key issues in the telerobotics control design. The com-munication characteristics between the master and slave systemare the crucial factors for the achievable transparency level intelerobotics control architectures [38]. Slight delay deterioratesthe realistic perception of the remote environment. In the futurestudy, some control algorithms should be adopted to compen-sate the hysteretic characteristics of the designed master hapticinterface.

VII. CONCLUSION

In this paper, the RAECS was presented to provide inter-ventionalists with the ability to use their conventional bedsidecatheterization skills in the robotic endovascular procedure. Thehaptic force feedback strategy was utilized to assist the surgeonin decision-making and improving catheter interventional skillsduring teleoperated catheter interventional practice. Experimen-tal results illustrated that the haptic force feedback was a benefitfor providing natural haptic sensation and reducing the humancognitive workload as well as maintaining the safety of surgery.According to the performance evaluation metrics, we also foundthat the challenge of improving system transparency in roboticendovascular surgery can be potentially addressed by adoptingcatheter-based haptic force feedback.

REFERENCES

[1] H. Rafii-Tari, C. J. Payne, and G. Z. Yang, “Current and emergingrobot-assisted endovascular catheterization technologies: A review,” Ann.Biomed. Eng., vol. 42, no. 4, pp. 697–715, 2014.

[2] L. W. Klein et al., “Occupational health hazards in the interventionallaboratory: Time for a safer environment,” Catheterization CardiovascularInterventions, vol. 73, no. 3, pp. 432–438, 2009.

[3] S. Au et al., “Robotic endovascular surgery,” Asian Cardiovascular Tho-racic Ann., vol. 22, no. 1, pp. 110–114, 2014.

[4] T. S. Lendvay, B. Hannaford, and R. M. Satava, “Future of robotic surgery,”Cancer J., vol. 19, no. 2, pp. 109–119, 2013.

[5] P. Kazanzides et al., “Surgical and interventional robotics-core concepts,technology, and design,” IEEE Robot. Autom. Mag., vol. 15, no. 2, pp. 122–130, Jun. 2008.

[6] S. Hirche and M. Buss, “Human-oriented control for haptic teleoperation,”Proc. IEEE, vol. 100, no. 3, pp. 623–647, Mar. 2012.

[7] A. M. Okamura, C. Basdogan, S. Baillie, and W. S. Harwin, “Haptics inmedicine and clinical skill acquisition,” IEEE Trans. Haptics, vol. 4, no. 3,pp. 153–154, Jul./Sep. 2011.

[8] K. Zareinia, Y. Maddahi, C. Ng, N. Sepehri, and G. R. Sutherland, “Perfor-mance evaluation of haptic hand-controllers in a robot-assisted surgicalsystem,” Int. J. Med. Robot. Comput.-Assisted Surgery, vol. 11, no. 4,pp. 486–501, 2015.

[9] C. Pacchierotti, M. Abayazid, S. Misra, and D. Prattichizzo, “Teleop-eration of steerable flexible needles by combining kinesthetic and vi-bratory feedback,” IEEE Trans. Haptics, vol. 7, no. 4, pp. 551–556,Oct./Dec. 2014.

[10] D. A. Lawrence, “Stability and transparency in bilateral teleoperation,”IEEE Trans. Robot. Autom., vol. 9, no. 5, pp. 624–637, Oct. 1993.

[11] X. Bao, S. Guo, N. Xiao, Y. Li, C. Yang, and Y. Jiang, “A cooperationof catheters and guidewires-based novel remote-controlled vascular in-terventional robot,” Biomed. Microdevices, 2018, to be published, doi:10.1007/s10544-018-0261-0.

[12] O. M. Omisore et al., “Towards characterization and adaptive compen-sation of backlash in a novel robotic catheter system for cardiovascularinterventions,” IEEE Trans. Biomed. Circuits Syst., vol. 12, no. 4, pp. 824–838, Aug. 2018.

https://dx.doi.org/10.1007/s10544-018-0261-0


[13] H. J. Cha, B. J. Yi, and J. Y. Won, “An assembly-type master–slavecatheter and guidewire driving system for vascular intervention,” in Proc.Inst. Mech. Eng., Part H, J. Eng. Med., 2017, vol. 231, no. 1, pp. 69–79.

[14] Y. Zhao, S. Guo, N. Xiao, Y. Wang, Y. Li, and Y. Jiang, “Operating forceinformation on-line acquisition of a novel slave manipulator for vascularinterventional surgery,” Biomed. Microdevices, 2018, to be published, doi:10.1007/s10544-018-0275-7.

[15] A. S. Shafer and M. R. Kermani, “Design and validation of a magneto-rheological clutch for practical control applications in human-friendlymanipulation,” in Proc. IEEE Int. Conf. Robot. Autom., 2011, pp. 4266–4271.

[16] X. Yin, S. Guo, H. Hirata, and H. Ishihara, “Design and experimental eval-uation of a teleoperated haptic robot assisted catheter operating system,”J. Intell. Mater. Syst. Struct., vol. 27, no. 1, pp. 3–16, 2016.

[17] D. Case, B. Taheri, and E. Richer, “Dynamical modeling and experimentalstudy of a small-scale magnetorheological damper,” IEEE/ASME Trans.Mechatronics, vol. 19, no. 3, pp. 1015–1024, Jun. 2014.

[18] W. Li, P. Yadmellat, and M. R. Kermani, “Linearized torque actuation us-ing FPGA-controlled magnetorheological actuators,” IEEE/ASME Trans.Mechatronics, vol. 20, no. 2, pp. 696–704, Apr. 2015.

[19] J. D. Carlson, D. M. Catanzarite, and K. A. St. Clair, “Commercialmagneto-rheological fluid devices,” Int. J. Modern Phys. B, vol. 23/24,pp. 2857–2865, 1996.

[20] Y. Song et al., “Design and performance evaluation of a haptic interfacebased on MR fluids for endovascular tele-surgery,” Microsyst. Technol.,vol. 24, no. 2, pp. 909–918, 2018.

[21] Y. Song et al., “Performance evaluation of a robot-assisted catheter op-erating system with haptic feedback,” Biomed. Microdevices, 2018, to bepublished, doi: 10.1007/s10544-018-0294-4.

[22] X. Yin, S. Guo, and Y. Song, “Magnetorheological fluids actuated haptic-based teleoperated catheter operating system,” Micromachines, vol. 9,no. 9, p. 465, 2018, doi: 10.3390/mi9090465.

[23] X. Yin, S. Guo, N. Xiao, T. Tamiya, H. Hirata, and H. Ishihara,“Safety operation consciousness realization of a MR fluids-based novelhaptic interface for teleoperated catheter minimally invasive neuro-surgery,” IEEE/ASME Trans. Mechatronics, vol. 21, no. 2, pp. 1043–1054,Apr. 2016.

[24] N. Xiao et al., “Clamping force evaluation for a robotic catheter navigationsystem,” Neurosci. Biomed. Eng., vol. 1, no. 2, pp. 141–145, 2013.

[25] X. Yin, S. Guo, and Y. Wang, “Force model-based haptic master consoledesign for teleoperated minimally invasive surgery application,” in Proc.IEEE Int. Conf. Mechatronics Autom., 2015, pp. 749–754.

[26] Y. Thakur, D. W. Holdsworth, and M. Drangova, “Characterization ofcatheter dynamics during percutaneous transluminal catheter procedures,”IEEE Trans. Biomed. Eng., vol. 56, no. 8, pp. 2140–2043, Aug. 2009.

[27] N. A. C. Zakaria, T. Komeda, and C. Y. Low, “Mechatronic design for afail-safe catheter guide system,” in Proc. Int. Mech. Eng. Congr. Expo.,2010, pp. 491–500.

[28] G. A. Gescheider, Psychophysics: The Fundamentals. Hove, U.K.: Psy-chology Press, 2013.

[29] X. D. Pang, H. Z. Tan, and N. I. Durlach, “Manual discrimination of forceusing active finger motion,” Attention, Perception, Psychophys., vol. 49,no. 6, pp. 531–540, 1991.

[30] N. Benard, D. Coisne, E. Donal, and R. Perrault, “Experimental studyof laminar blood flow through an artery treated by a stent implantation:Characterisation of intra-stent wall shear stress,” J. Biomechanics, vol. 36,no. 7, pp. 991–998, 2003.

[31] T. Wang, D. Zhang, and L. Da, “Remote-controlled vascular interventionalsurgery robot,” Int. J. Med. Robot. Comput.-Assisted Surgery, vol. 6, no. 2,pp. 194–201, 2010.

[32] E. Maor, M. F. Eleid, R. Gulati, A. Lerman, and G. S. Sandhu, “Currentand future use of robotic devices to perform percutaneous coronary in-terventions: A review,” J. Amer. Heart Assoc., vol. 6, no. 7, 2017, Art.no. e006239.

[33] G. Srimathveeravalli, T. Kesavadas, and X. Li, “Design and fabrication ofa robotic mechanism for remote steering and positioning of interventionaldevices,” Int. J. Med. Robot. Comput.-Assisted Surgery, vol. 6, no. 2,pp. 160–170, 2010.

[34] Y. Thakur, J. X. Bax, D. W. Holdsworth, and M. Drangova, “Design andperformance evaluation of a remote catheter navigation system,” IEEETrans. Biomed. Eng., vol. 56, no. 7, pp. 1901–1908, Jul. 2009.

[35] H. Rafii-Tari, J. Liu, S. L. Lee, C. Bicknell, and G. Z. Yang, “Learn-ing based modeling of endovascular navigation for collaborative roboticcatheterization,” in Proc. Int. Conf. Med. Image Comput. Comput.-AssistedIntervention, 2013, pp. 369–377.

[36] M. K. O’Malley, A. Gupta, M. Gen, and Y. Li, “Shared control in hapticsystems for performance enhancement and training,” J. Dyn. Syst., Meas.,Control, vol. 128, no. 1, pp. 75–85, 2006.

[37] H. Boessenkool et al., “A task-specific analysis of the benefit of hapticshared control during telemanipulation. haptics,” IEEE Trans. Haptics,vol. 6, no. 1, pp. 2–12, Jan/Mar. 2013.

[38] S. Hirch and M. Buss, “Human perceived transparency with time delay,”Advances in Telerobotics. Berlin, Germany: Springer, 2007, pp. 191–209.

Shuxiang Guo (SM’03) received the Ph.D. degree inmechanoinformatics and systems from Nagoya Uni-versity, Nagoya, Japan, in 1995.

He is currently a Full Professor of Robotics withthe Department of Intelligent Mechanical System En-gineering, Kagawa University, Takamatsu, Japan. Heis also the Chair Professor with the Key Labora-tory of Convergence Medical Engineering Systemand Healthcare Technology, Ministry of Industry andInformation Technology, Beijing Institute of Tech-nology, China. He has published about 470 refereed

journal and conference papers. His current research interests include biomimeticunderwater robots and medical robot systems for minimal invasive surgery, mi-cro catheter system, micropump, and smart material (SMA, IPMC) based onactuators.

Dr. Guo is an Editor-in-Chief for the International Journal of Mechatronicsand Automation.

Yu Song received the Ph.D. degree in intelligent me-chanical systems engineering from Kagawa Univer-sity, Takamatsu, Japan, in 2018.

He is currently a Lecturer with the School of Elec-trical and Electronic Engineering, Tianjin Universityof Technology, Tianjin, China. His current researchinterests include medical robotics, haptic control,human robot interaction, teleoperation system, andartificial intelligence.

Xuanchun Yin received the Ph.D. degree in intelli-gent mechanical systems engineering from KagawaUniversity, Takamatsu, Japan, in 2016.

He is currently a Senior Lecturer with the Depart-ment of Automation Engineering, South China Agri-cultural University, Guangzhou, China. His currentresearch interests include haptics, human robot in-teraction, teleoperation system, artificial intelligence,soft robotics, bio-inspired robotics, medical robotics,agricultural robotics, and soft actuator design.

Linshuai Zhang received the M.S. degree in me-chanical manufacturing and automation from theChangchun University of Science and Technology,Changchun, China, in 2015. He is currently work-ing toward the Ph.D. degree in intelligent mechan-ical systems engineering with Kagawa University,Takamatsu, Japan.

His current research interests include medicalrobot systems for minimal invasive surgery.

https://dx.doi.org/10.1007/s10544-018-0275-7

https://dx.doi.org/10.1007/s10544-018-0294-4

https://dx.doi.org/10.3390/mi9090465


Takashi Tamiya received the Ph.D. degree in neuro-logical surgery from the Medical School, OkayamaUniversity, Okayama, Japan, in 1990.

He had fellowships with Massachusetts GeneralHospital, Harvard Medical School in the USA from1993 to 1994. He is currently a Full Professor ofNeurological Surgery with the Faculty of Medicine,Kagawa University, Takamatsu, Japan. He has pub-lished more than 400 refereed journal and conferencepapers. His current research interests include the sur-gical techniques of neurosurgical operations and in-

travascular surgery system.

Hideyuki Hirata received the B.S. and M.S. de-grees from Yokohama National University, Yoko-hama, Japan, in 1981 and 1983, respectively, and thePh.D. degree from the Tokyo Institute of Technology,Tokyo, Japan, in 1992, all in mechanical engineering.

He is currently Professor of Intelligent MechanicalSystems Engineering with Kagawa University, Taka-matsu, Japan. His current research interests includematerial strength, material design, and microdevicesbased on computer simulation technology.

Hidenori Ishihara received the B.S., M.S., and Ph.D.degrees in electrical machinery from Nagoya Uni-versity, Nagoya, Japan, in 1991, 1993, and 1995,respectively.

He is currently an Associate Professor of Intelli-gent Mechanical Systems Engineering with KagawaUniversity, Takamatsu, Japan. His current researchinterests include intellectualization and functional-ization of robotics.

A Novel Robot-Assisted Endovascular Catheterization System ...guolab.org/Papers/2019/song...

Documents

Transcript of A Novel Robot-Assisted Endovascular Catheterization System ...guolab.org/Papers/2019/song...