Mechatronics 56 (2018) 115–131

Contents lists available at ScienceDirect

Mechatronics

journal homepage: www.elsevier.com/locate/mechatronics

Review

Development of an integrated haptic system for simulating upper

gastrointestinal endoscopy

Shanthanu Chakravarthy

∗ , Mythra V.S. Balakuntala , Ashwin M. Rao , Ravi Kumar Thakur , G.K. Ananthasuresh

Department of Mechanical Engineering, Indian Institute of Science, Bangalore, India

a r t i c l e i n f o

Keywords:

Medical simulator Haptics Endoscopy

a b s t r a c t

Virtual reality together with haptics offers immersive and flexible platforms for training doctors in medical proce- dures. In this paper, we present mechanical design, control, integration, and user-studies of a virtual reality-based haptic simulator for Upper Gastrointestinal (GI) endoscopy. The design overcomes some of the limitations of the existing systems. First, there is an extra degree of freedom for simulating junctions controlled by sphincter mus- cles in addition to translational and rotational degrees of freedom along and about the axis of the endoscope-tube. Second, the force-feedback is continuous over a longer range in all three degrees of freedom. Third, multiple inser- tions and removals of the tube are made possible with a magnetically actuated snap-fit mechanism. A Dynamics- based feed-forward control algorithm is developed and characterized for fidelity and transparency. The system

provides continuous force up to 11 N in the axial direction and continuous torque up to 255 mN.m about the axial direction. The haptic device is integrated with a physics-based Virtual Reality (VR) interface. Furthermore, an immersion study was conducted using the integrated system with a cohort of novice and experienced clini- cians. The haptic response and virtual model were rated high and improvements were suggested for graphical visualization and physical arrangement.

1

b

a

c

o

i

t

d

t

i

p

o

d

n

m

b

s

f

r

m

c

a

b

H

t

f

d

i

1

d

T

t

T

fi

t

s

hRA0

. Introduction

Upper gastrointestinal (GI) endoscopy involves insertion of a flexi-le scope called an endoscope into the digestive tract for medical ex-mination and, of late, for surgical procedures. GI endoscopy, includingolonoscopy, is a complex task involving a high degree of eye-hand co-rdination [1] . Therefore, in order to practice endoscopy safely, a train-ng system that does not involve patients is deemed important. That is,he training simulator should provide the same experience of a real en-oscopy procedure in the absence of a patient. In this paper, we discusshe mechatronic development, which includes design, prototyping, andntegration, of a Virtual Reality (VR)-based simulator for training in up-er GI endoscopy. We also present and discuss immersion studies carriedut with practicing doctors. Fig. 1 shows the laboratory prototype of theeveloped endoscopy simulator.

VR-based training devices have many advantages [2] . They are eco-omical in the long run and usable any number of times. The trainingodel can be changed and designed as per requirements including those

ased on real clinical cases. They can be used for quantitatively assessingkill. Furthermore, recorded training sessions could be used for identi-ying deficiencies and positive traits of the trainees. However, the cur-

∗ Corresponding author. E-mail address: shanthanuc@iisc.ac.in (S. Chakravarthy).

ttps://doi.org/10.1016/j.mechatronics.2018.10.006 eceived 17 April 2017; Received in revised form 26 August 2018; Accepted 19 Octovailable online 12 November 2018 957-4158/© 2018 Elsevier Ltd. All rights reserved.

ent VR-based endoscopy training devices do not provide complete im-ersion [ 3 , 4 ]. Until recent times, VR-based endoscopy training systems

oncentrated mostly on visual feedback using medical image processingnd graphical rendering techniques. However, in addition to visual feed-ack, it is important for the clinicians to feel the interaction forces [5] .aptic feedback is known to greatly enrich the user-experience during

raining [6] . Furthermore, kinematics and dynamics of the user inter-ace and those in the virtual environment may be different. A hapticevice acts as an interface for creating transparent and more realisticnteraction than what is possible in the graphics-only interface [7] .

.1. Review of related work

Commercially available endoscopy training devices and the en-oscopy haptic devices reported in the literature are reviewed inable 1 . The existing endoscopy simulators have one or more limita-ions. Majority of the designs use spring-loaded friction drives [8–13] .hey are compact. They apply brake and use friction to achieve suf-cient gripping force. Such friction-based designs are difficult to con-rol. This leads to low transparency leading to undesirable forces duringimulation [ 8 , 12 , 14 ]. They also have difficulty in simulating decoupled

ber 2018

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 1. Laboratory prototype of the haptic device developed at IISc, Bengaluru.

Table 1

Review of existing endoscopy simulators.

Endoscopy simulator Remarks on haptic design Reported characteristics

Continuous force (N)

Continuous torque (mN.m)

Weight (kg) Bandwidth (Hz)

Ikuta et al. ○ Design based on friction cylinders/roller 8 85 4.5 n/a ○ Difficult to control and forces are not

reliable

Korner and Manner ○ Generates tip force using a belt drive 20 n/a n/a n/a ○ Does not allow for multiple insertion and

removal of scope

Ilic et al. ○ Friction belt and differential gear 44 2200 n/a n/a ○ Drive system with high inertia and friction

Woo et al./Yi et al. ○ Uses folding guides moving on rails 7.15 40 > 80 6.5 ○ Force sensor in feedback loop ○ Multiple insertion and removal is not

possible

Samur et al. ○ Uses V-type friction rollers and custom

brakes 5.4 85 n/a 13

○ Compact design ○ Slip in rotary direction

ENDOVR (CAE healthcare) ○ Endoscope guided around a cylinder/drum n/a n/a n/a n/a ○ One DoF design

GI Mentor (3D systems) ○ No active force-feedback n/a n/a n/a n/a ○ Instantaneous force through balloon

inflation

ENDOSIM (Surgical science) ○ V-roller for friction drive n/a n/a n/a n/a ○ The design is derived from [10]

Dargar et al. ○ Omnidirectional friction wheel 5.6 190 n/a 18 ○ No insertion/removal feature

This work ○ Longitudinal rail with capstan drive 11 255 9 40 ○ Additional DoF for sphincter simulation

f

i

d

t

a

t

c

p

b

t

p

t

s

n

m

o

n

a

orces, i.e., gripping or braking in one DoF can lead to undesirable forcen another DoF. This type of design has the problem of slip [ 8 , 10 ]. Ad-itionally, haptic torque in the rotary direction is limited [ 10 , 12 ]. Fur-hermore, wear of the expensive endoscope tube used in simulation isn issue with friction-based designs.

Some other endoscopy haptic devices use a guide for distal end ofhe endoscope-tube [15–17] . The end of the tube is guided either on aylindrical drum [18] or a linear rail. Although such designs are sim-le and have decouple large longitudinal and rotational DoFs, they areulky and have limited range of motion. They have large inertia due

116

o moving guides [ 16 , 18 ]. They use belt drives and gears that pose theroblem of backlash and alignment. Furthermore, as the distal end ofhe endoscope must be fixed to the guide [ 16 , 17 ], there is no provi-ion for multiple insertions and removals of the endoscope. It may beoted the provision for multiple insertions and removals is important forastering the skill of intubation during training [ 10 , 19 ]. We designed

ur system to overcome the limitations of existing systems, as explainedext.

The proposed design uses a low-friction linear rail-guide to provide continuous force normal to the distal end of the scope. A carriage on

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 2. Block diagram of endoscopy simulator.

t

t

i

t

t

t

t

t

e

n

t

l

D

t

u

T

t

i

e

F

r

r

2

t

c

c

v

a

a

i

p

t

r

e

t

t

t

T

t

t

p

w

f

f

O

c

t

T

c

e

c

t

c

b

v

o

i

n

t

F

i

V

g

i

v

G

l

m

t

r

a

m

c

o

s

f

o

a

o

d

t

S

s

a

s

r

r

t

t

n

f

f

t

s

i

m

he rail-guide is driven by a capstan drive and a rotary motor fixed tohe base. This arrangement reduces backlash and inertia. Rotary torques simulated by a motor attached to the carriage. The rotary motor onhe carriage drives a shaft assembly that engages with the distal end ofhe endoscope. A magnetically actuated snap-fit mechanism attached tohe shaft assembly is used to passively engage and disengage the dis-al end of tube with the rail-guide. The snap-fit mechanism is designedo transfer rotary torque and longitudinal force to the distal end of thendoscope without slip. The design features such as reduced friction,o backklash, transmission of force without slip are combined with ac-ive feedback control to achieve high fidelity and transparency in simu-ating force. Furthermore, the haptic interface comprises an additionaloF in the radial direction. The radial mechanism simulates sphinc-

er forces in endoscopy. The radial DoF is particularly useful to sim-late forces for entry into the narrow upper esophageal junction [20] .his simulation is important because the sphincters are critical junc-ions and training has to be incorporated to avoid accidental punctur-ng [19] .

The endoscopy simulator is introduced in Section 2 . The design el-ments of the force-reflecting mechanism are presented in Section 3 .orce control and VR-interface are presented in Section 4 and Section 5 ,espectively. Immersion studies in Section 6 are followed by concludingemarks in Section 7 .

. Integrated endoscopy simulator

Fig. 2 shows a block diagram of the endoscopy simulator along withhe flow of information in the system. The simulator consists of fouromponents: a simulation endoscope, a force-reflecting system, a forceontroller, and a VR system for computing the interaction forces forisualization and haptic feedback.

The simulation endoscope is illustrated in Fig. 3 (a). The user inter-cts with the endoscopy simulator through the force-reflecting mech-nism and the simulation endoscope. The force-reflecting mechanisms a mechanical system together with actuators and sensors for im-arting kinesthetic force sensations to the user. During endoscopy,he important maneuvers can be decomposed into longitudinal andotational DoF of the endoscope. As shown in Fig. 3 (b), the distalnd of the endoscope interacts with the stomach. The interaction athe distal end results in longitudinal force along the axis and ro-ational torque about the axis of the endoscope. These forces andorques are felt at the user’s hand through the simulation endoscope.he endoscopy haptic device is designed to simulate these interac-ion forces. Fig. 3 (a) illustrates the simulation of interaction forces athe user end of the endoscopy haptic device. Fig. 4 shows the com-lete computer model of the endoscopy haptic device developed in thisork.

A force controller is developed to impart desired force through theorce-reflecting mechanism. The torque from the DC motor drives theorce-reflecting mechanism to impart controlled forces onto the user.ur force controller uses dynamics-based feed-forward terms and motorurrent measurements for feedback. The knob rotations on the simula-ion endoscope and user-interactions are tracked by the force controller.

117

he force control system communicates position to the VR interface andontrols the forces imparted to the user. For ease of development andxperiments, the control algorithms are implemented on a dSPACE 1103ontrol prototyping board running at 1 kHz.

The VR interface provides visual stimulus ( Fig. 3 (c)) and computeshe interaction physics for the endoscopy simulation. This system re-eives the position and velocity information from the force controllerlock. The interaction of the simulation endoscope is updated in theirtual environment of the VR-interface. Rotation of the control knobsn the simulation endoscope that leads to different graphical scenes isllustrated in Fig. 3 (d). The position information is used to compute theecessary reaction forces and global deformations. Our force computa-ion model is physics-based and employs Finite Element Analysis (FEA).urthermore, the VR interface also uses graphical techniques for real-stic rendering of graphical scenes. The VR interface is implemented inC ++ and runs on a Windows operating system.

Important design requirements for the endoscopy simulator can berouped into three categories: physical characteristics, haptic character-stics, and VR-simulation characteristics. Physical characteristics pro-ide for important face validity of the training simulator [21] . UpperI-endoscopy begins at the mouth and generally ends at the duodenum

ocated at around 0.8 m from the mouth in an average adult. Further-ore, scope rotation or torquing motions are commonly used by doc-

ors during an endoscopic session [19] . Our design provides for infiniteotation and longitudinal motion up to one meter. The design also en-bles multiple insertion and removal of scope through a passive snap-fitechanism. The complete drive system, actuator, and sensor unit are ac-

ommodated within a space of 1100 ×180 ×200 mm

3 . The dimensionsf the haptic interface allow for encasing all the mechanical and driveystems completely behind a mannequin head. This dimension providesor face validity through hardware realism. Furthermore, the prototypef the haptic interface developed in this work weighs less than 10 kgnd is easily transportable.

Studies show that during colonoscopy, only 5% of forces are in excessf 10 N and the maximum force is about 17 N [ 22 , 23 ]. Similarly, torqueuring common colonoscopy procedures is below 200 mN.m. However,here is no literature available for forces during an upper GI procedure.ince the upper GI has no sharp bends as in the sigmoid colon, it isafe to assume that forces required for simulating upper GI endoscopyre also less than 10 N. Furthermore, it is reported that a minimumtiffness of 0.2 N/m or a reaction force of 10 N is enough to simulateigid objects [ 24 , 25 ]. Our design can simulate continuous force in theange of ± 11 N and a continuous torque range of ± 200 mN.m. In addi-ion to the requirements imposed by the task, the design should meethe desirable haptic features such as high transparency [24] , low sig-al loss [26] (fidelity > 0.9), and stiffness [7] . By using model-basedeed-forward control, we increase transparency and reduce forces inree space simulation to below 0.2 N. Furthermore, a bandwidth morehan human hand interaction speed of 10 Hz is achieved in our de-ign [27] .

It is reported that [7] for stable haptic rendering, simulation ratesn excess of 1 kHz is necessary. However, visualization can run atuch lower speeds. A minimum of 30 Hz is required for smoothness in

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 3. (a) Illustration of interaction forces in real endoscopy. (b) Forces at the user end of the force-reflecting mechanism. (c) Graphical rendering of interaction scene. (d) Illustration of changing graphical scene with control knob inputs.

Fig. 4. Computer model of the force reflecting mechanism.

g

g

a

t

r

i

o

a

a

3

3

fi

n

t

c

raphical rendering. We use Finite Element (FE)-based modeling to-ether with pre-computation to simulate physically accurate simulationst more than 1 kHz rate. To achieve accurate anatomical representation,he upper GI model is constructed from visible human data [28] and isefined with practicing endoscopists. All the desired specifications formmersive and the immersive endoscopy simulation platform is devel-ped with clinical guidance from practicing endoscopists and the avail-ble literature. A table in the concluding section summarizes the desirednd achieved characteristics of the developed endoscopy simulator.

118

. Force-reflecting mechanism

.1. Design exploration

Conceptual design exploration was carried out before arriving at thenal design of the haptic device shown in Fig. 4 . In addition to longitudi-al guide-based design (Concept 3), two other designs, one based on fric-ion rollers (Concept 1) and the other based on the helical cylinder (Con-ept 2) were explored. Preliminary bench-top prototypes of the designs

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 5. (a) Endoscopy haptic design based on friction rollers. (b) design based on helical cylinder.

Table 2

Concept scoring matrix.

Concept 1 Concept 2 Concept 3

Selection criteria Weight (%) Rating Weighted score Rating Weighted score Rating Weighted score

Compactness 10 5 0.5 4 0.4 2 0.2 Multiple insertion 10 5 0.5 2 0.2 2, 5 a 0.2 Durability 5 2 0.1 3 0.15 4 0.2 Ease of manufacturing 20 3 0.6 2 0.4 4 0.8 Range 15 5 0.75 4 0.6 3 0.45 Decoupling 15 3 0.45 3 0.45 4 0.6 Haptic design features- Inertia and friction 25 3 0.75 3 0.75 4 1

Total score 3.65 2.95 3.45, 3.75 a

Rank 1 3 2

a Concept 3 with a snap-fit mechanism added to allow multiple insertion/removal.

w

m

d

r

t

H

a

t

s

i

w

0

t

w

c

f

2

t

r

f

u

h

t

r

f

g

w

3

s

a

a

D

c

D

t

d

s

M

i

ere developed for a proof-of-concept study. Fig. 5 shows the computerodels and the fabricated bench-top prototypes. Friction roller basedesign is similar to [10] . The endoscope is passed through the frictionollers for simulating translational DoF. Rotary DoF is achieved by ro-ating the entire longitudinal DoF assembly. This adds to the inertia.elical coil design is a novel design and involves coiling the endoscoperound a grooved helical cylinder. The coiling is assisted by a motorhat drives the lead-screw attached to the helical coil. Rotational DoF isimulated by rotating the entire assembly using a direct drive motor.

Design Concepts 1 and 2 are compact. However, both have highnertia in the rotary direction. In Concept 1, slip and gripping issuesere noticed. For an endoscope, which has a coefficient of friction of.28, about 40 N gripping force is required. This results in the large mo-or for longitudinal DoF, which adds to the inertia. Design 1 also hasiring complications due to rotating longitudinal DoF. In design Con-

ept 2, a motor with 300 mN.m torque is required for the continuousorce of ± 10 N in the longitudinal DoF. Furthermore, design Concept requires the translational motors to be sliding freely to wind aroundhe helical groove of the cylinder. This introduces complexity for theotary DoF.

The three design concepts considered in this work were evaluatedor important design criteria. Table 2 summarizes the evaluation done

119

sing concept scoring matrix [29] . Concept 1 based on friction rollersad a high score and was ranked 1. However, we considered the facthat Concept 3 could be improved by facilitating multiple insertion andemoval. It also gave advantages in terms of construction and hapticeatures. Considering these factors, Concept 3 based on the longitudinaluide was selected for complete development. The complete design andorking are explained next.

.2. Longitudinal DoF

Fig. 6 illustrates the longitudinal DoF of the developed endoscopyimulator. Upper GI-endoscopy begins at the mouth and generally endst the duodenum located at around 0.8 m from the mouth in an aver-ge adult. To account for this workspace requirement, the longitudinaloF is achieved by a carriage C moving on a 1 m-long rail-guide R. Thearriage is actuated by a specially designed longitudinal capstan drive shown in Fig. 6 . The capstan drive cable is attached on either side of

he carriage through a distinct screw-driven mechanism. As the screw-riven mechanism is tightened, it pulls the capstan cable from eitheride. This allows for tensioning the cable after assembly. A RE seriesaxon DC motor (RE 40 448590) with a maximum torque of 88.2 mN.m

s used as the actuator for the longitudinal capstan drive. When the mo-

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 6. Translational capstan drive system for longitudinal motion of the carriage.

t

t

t

o

t

c

T

I

m

(

a

s

c

w

a

m

r

a

n

c

o

b

p

w

i

t

b

v

w

c

t

t

𝐽

fi

e

𝐹

w

f

m

𝜏

w

s

c

𝑚

w

t

t

c

3

d

o

l

p

f

b

n

o

t

m

a

s

f

n

w

p

s

c

c

i

o

i

T

a

l

t

u

or shaft rotates, it winds the wire on it from one side and unwinds fromhe other causing the carriage to translate along the rail-guide. The mo-or has a maximum speed of 4 m/s and can impart a maximum forcef 11.8 N. Sensing of the displacement happens through an encoder at-ached to the motor. It has 1024 Counts Per Revolution (CPR). Thisorresponds to a resolution of 0.047 mm in the longitudinal direction.he construction is made simple by attaching an encoder to the motor.f the encoder is attached anywhere else, for example, on the pulleys, itight give incorrect measurements due to slip.

To carry the load of the carriage and to reduce friction, a rail-guideRollon Bearings, RB 28) is employed for longitudinal translation. It is heavy-duty rail-guide consisting of a slider with radial bearings thatlide on the internal raceways of the carbon steel rail profile, which isold-drawn and zinc-plated. It allows for a maximum speed of 5 m/s,hich is adequate for the intended use. The distal end of the endoscopettaches to a snap-fit mechanism of the carriage. The role of the snap-fitechanism is explained in the next section.

The endoscope is a flexible tube and tends to buckle when the car-iage applies longitudinal force at the distal end. To restrict this, a slottedcrylic pipe is used to enclose the endoscope (see Fig. 4 ). The longitudi-al slot in the pipe is necessary to allow the unhindered traversal of thearriage motor along its length. One-meter-long acrylic pipe of 25 mmuter diameter and 5 mm thickness was chosen based on the fixed-fixedeam model to ensure adequate stiffness.

The standard equation of motion for the mechanical and electricalarts of the motor driving the translational DoF can be written as [30]

𝐽 𝑚

𝐾 𝑡

𝑅 𝜙 +

(

𝑏 𝑑

𝐾 𝑡

𝑅 + 𝐾 𝑣

)

�� = 𝑉 ( 𝑡 ) (1)

here J m

is the motor rotary inertia and b d is the damping coefficientn the motor. K t ,K v , and R are the motor electrical parameters namely,orque constant, speed constant, and resistance respectively. The sym-ol 𝜙 denotes the angular position of the motor shaft and V the inputoltage.

The endoscopy haptic device is an impedance-type haptic devicehere force control is achieved by controlling the motor torque. We

ontrol V ( t ) in Eq. (1) to give the required motor torque using the rela-ions, 𝑉 ( 𝑡 ) = 𝑖 ( 𝑡 ) 𝑅 and 𝜏𝑚 = 𝐾 𝑡 𝑖 ( 𝑡 ) . Also adding the torque 𝜏c due tohe carriage moving on the rail guide, we have:

𝑚 �� +

(

𝑏 𝑑 +

𝐾 𝑡 𝐾 𝑣

𝑅

)

�� + 𝜏𝑐 = 𝜏𝑚 (2)

The preceding equation gives the torque for the DC motor. Simpli-ed dynamics of the carriage C that moves on the rail guide R can bexpressed as

= 𝑚 𝑥 + 𝑓 (3)

𝑐 𝑟

120

here x is the longitudinal position, m the mass of the carriage, and r the Coulomb friction in the system. By reflecting this force onto theotor shaft, we get

𝑐 = 𝑟 2 𝑚 𝜙 + 𝑟 𝑓 𝑟 (4)

here r is the radius of the capstan driver attached to the motor. Afterubstituting (4) in (2) , the equation of motion for the equivalent systeman be expressed as

𝑒 1 �� + 𝑏 𝑒 1 �� + 𝜏𝑒 1 = 𝜏𝑚 1 (5)

here 𝑚 𝑒 1 = 𝐽 𝑚 + 𝑟 2 𝑚 is the equivalent mass, and 𝑏 𝑒 1 = 𝑏 𝑑 +

𝑘 𝑡 𝑘 𝑣

𝑅 is

he equivalent damping in the system, and 𝜏𝑒 1 = 𝑟 𝑓 𝑟 is the equivalentorque due to friction. These dynamic terms are used in feedforwardontrol for compensating undesired dynamics.

.3. Snap-fit mechanism

For the longitudinal and rotational forces to be transferred to the en-oscope, the endoscope tip must be attached to the shaft of the motorn the carriage such that there is no relative motion, rotational or trans-ational, between the endoscope tip and the motor shaft. The endoscopyrocedure requires frequent removal and reinsertion of the endoscoperom the mouth of the patient [10] . Due to this, the endoscope tip cannote permanently attached to the shaft of the motor. Therefore, a mecha-ism is designed to allow the endoscope tip to be attached to the motorn the carriage during insertion of the endoscope. It also facilitates de-achment during removal of the endoscope.

The use of an additional actuator and power supply to actuate aechanism would result in unnecessary weight on the motor shaft. This

lso leads to complications in wiring and control. Therefore, a passivenap-fit mechanism is designed to be actuated naturally with minimumorce during insertion of the endoscope. The designed snap-fit mecha-ism is shown in Fig. 7 . The mechanism has two parts. A socket thatould be fixed to the shaft moving inside the slotted pipe. The secondart is a plug, which would be fixed to the tip of the endoscope. Theocket consists of four flexible strips/beams (see Fig. 7 (a)) arranged cir-umferentially together with guides to ensure that the plug orients itselforrectly during insertion. Magnets are arranged circumferentially, bothnside the plug and inside the flexible strips of the socket. The magnetsn the plug attract the flexible strips of the socket. This causes the flex-ble beams of the socket to bend and latch onto the plug (see Fig. 7 (b)).he design of the flexible beams and the plug is such that the latchingrrests both translational and rotational motion.

Another requirement for the snap-fit mechanism is the natural re-ease of the mechanism when the endoscope is pulled back to the en-rance of the haptic device. Similar to snap-fit latching, magnets aresed in the design for releasing the latch of the snap-fit mechanism. As

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 7. Snap-fit mechanism.

s

m

m

fl

c

p

n

m

f

p

h

T

fi

h

s

s

f

(

o

f

b

b

3

s

o

r

s

t

i

p

a

t

o

a

t

d

I

C

s

i

s

𝑚

w

t

d

3

m

c

d

r

s

A

b

w

b

T

m

r

i

t

b

t

m

r

4

r

hown in Fig. 7 (d) a funnel is designed at the entrance of the endoscopyechanism. The funnel consists of cylindrical magnets that attract theagnet embedded in the flexible strips of the socket. This causes theexible beams to move up and unlatch the snap-fit mechanism. To fa-ilitate alignment, the funnel is housed inside a bearing. The bearingrovides a passive way of fast alignment causing the cylindrical mag-ets to be aligned with magnets of the flexible strips. Thus, ensuringaximum force for deformation of the flexible strips. The fabricated

unnel housing is shown in Fig. 7 (e). Due to their complicated geometry, the plug and socket are 3D-

rinted (see Fig. 7 (c)). The flexible strips are designed such that theyave a maximum deformation of 3 mm with nearly 2 N magnetic force.his deformation of each of the four beams causes latching of the snap-t mechanism. The geometry of the flexible strips was designed toave a low stiffness to perform its function and still have adequatetrength to prevent breakage. COMSOL Finite Element Analysis (FEA)oftware was employed to refine the geometry to get the desired de-ormation. FEA simulations were carried out for VeroBlack material E = 1500 MPa, = 0.3). For unlatching, the powerful cylindrical magnetsn the funnel attract the magnets present on the flexible strips with aorce of 5 N. This produces more than 3 mm deformation of the flexi-le strips away from the plug. This deformation releases the latchingetween the socket and the plug attached to the endoscope.

.4. Rotational DoF

The rotational DoF is provided by the hollow aluminum shaft ashown in Fig. 8 . The shaft is driven by a belt drive through a motorn the carriage. The snap-fit is designed with square inserts that giveotary coupling between the endoscope and snap-fit, which is in turnocket attached to the shaft. The snap-fit and the shaft are designedo move inside the slotted acrylic pipe. However, the carriage carry-ng the motor is outside. This allows for using smaller diameter slottedipe to restrict complete buckling of the endoscope. To keep the drivessembly simple and still be able to provide high torque, a geared mo-or, and a belt drive is used. The system provides a maximum torquef 255 mN.m. Using the drive system will give rise to the backlash ofbout 0.7°. However, this is a design compromise to achieve high torquehat is not possible with a direct drive motor [31] . The design keeps the

121

rive system compact and reduces the inertia of the longitudinal DoF.t provides for unlimited rotational DoF. Rotation is sensed by the 1024PR encoder attached directly to the motor. The resolution of rotaryensing is 0.09°.

The dynamic equation for the rotary DoF is similar to Eq. (2) andncludes transmission ratio. The equation of motion for the equivalentystem is given by

𝑒 2 �� + 𝑏 𝑒 2 �� + 𝜏𝑒𝑓2 = 𝜏𝑒𝑚 2 (6)

here m e 2 is the equivalent mass and b e 2 is the equivalent damping inhe system. 𝜏em 2 and 𝜏ef 2 are the equivalent motor torque and torqueue to friction.

.5. Radial DoF

The radial DoF consists of a circumferentially actuated compliantechanism [20] . The circumferential compliant mechanism emulates

ontrolled and responsive circularly shaped opening. The mechanism isesigned in view of the anatomy of the throat and the special maneuverequired for intubation of the endoscope. The mechanism consists ofpecially designed planar beams. As shown in Fig. 9 (a), one end of beam is fixed to the outer ring and another end is connected to one end ofeam B to form the gripping pad. The other end of beam B is anchoredith a pin-joint. The beam assembly is cast in a polar array with fouream assemblies to form circumferentially distributed gripping pads.o restrict axial drift and allow for pure circumferential motion, theechanism is housed in a deep-groove ball-bearing assembly. The outer

ace of the mechanism is fixed to the front plate of the endoscopy hapticnterface as shown in Fig. 9 (b). A capstan drive system is used to drivehe circumferential compliant mechanism. A RE 30 brushed DC motory Maxon Motors is used to apply torque to the radial mechanism. Afterhe amplification with a capstan drive, the radial DoF can produce aaximum torque of 442 mN.m, which corresponds to more than 5 N

adial force on the endoscope.

. Force control

Transparency in a haptic device is indicative of the accuracy in forceendering. Free-space should feel free and the user should not feel the

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 8. Computer model and the fabricated design for the rotational DoF.

Fig. 9. (a) Circumferentially actuated compliant mechanism (b) Radial DoF assembly with a capstan drive and circumferential mechanism.

Fig. 10. Block diagram for the dynamic feed-forward control.

d

p

t

d

c

o

t

t

b

t

t

m

e

b

fc

H

p

t

t

ynamics of the device [ 32 , 33 ]. However, in open-loop control, com-uted voltage based on position is used to drive the desired currenthrough the motors. This type of control strategy lacks transparencyue to uncompensated dynamics of the system. Furthermore, the un-ertainties in the system causes the controller to fail at high-frequencyperation. We designed an impedance controller that compensates forhe device dynamics.

Our control strategy is based on feedback from measured current ofhe motor together with a feed-forward loop [34] . Fig. 10 shows thelock diagram of the impedance controller used in this work. The mo-or torque constant K t establishes the relation between the current and

122

orque provided by the motor. We measure the current in each of theotors and thus indirectly estimate the force output of the device. The

rror e between the desired force F d and the measured force F m

is fedack to an error-based controller. The controller maintains the desiredorce at the output of the system. Dynamics-based feedforward term F f ompensates the static friction and the inherent dynamics of the system.ere, F f is the desired motor torque computed my substituting measuredosition and velocities in Eqs. (5) and (7) . This compensation allows forhe use of lower gains and improves the stability of the overall system.

The dynamic equations for the feed-forward terms were derived inhe previous section. Table 3 gives the values of the important parame-

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 11. Force required to move in free space (a) without compensation (b) with compensation term in the control loop.

Table 3

Parameters used in the model.

Parameter Longitudinal Rotational

J m 119 10–7 kgm

2 35.9 10–7 kgm

2

K t 82.2 mN.m/A 19.9 mN.m/A K v 116 rpm/V 479 rpm/V R 2.49 Ω 0.378 Ωm 0.3 kg Na F c 2.2 N 1.4 N b d 20 mN.ms/rad 14 mN.ms/rad r 7.5 10–3 m Na

t

t

c

𝑓

w

ea

i

a

a

c

[

c

d

t

𝑢

w

b

s

r

s

𝑠

t

g

D

c

l

4

b

b

g

w

w

s

i

p

s

t

t

f

l

u

I

t

s

4

a

l

o

p

n

a

t

F

i

c

d

c

r

F

p

i

d

t

d

F

c

h

ers used in the dynamic model. Except for friction, all the values wereaken from the datasheet of the Maxon motors. For the friction term, aontinuous form of static friction model is used.

𝑟 ( ��) = 𝐹 𝑐 tanh ( ��) (7)

here F c is the Coulomb friction force. Its value was determined usingxperiments with the force sensor. The total damping in the system is f r dded to b e , which was introduced earlier.

The feedback force is measured indirectly by measuring the currentn the motors, which is in turn done by monitoring the voltage dropcross the resistors in the driver circuit. A first-order low-pass filter with time constant of 10 ms was implemented to eliminate noise in theurrent measurements.

To improve robustness, we employ a sliding mode control strategy35] . The sliding mode controller is robust in the sense that it gives ex-ellent tracking performance even with model inaccuracies, unmodeledynamics, and unknown external disturbance. The continuous form ofhe sliding mode control used in this work is given by

= 𝛽 tanh (𝑠

𝜀

)(8)

here 𝛽 is the controller gain that guarantees robustness and ɛ is a num-er that imposes bounds on the sliding manifold. In our method, theliding manifold s is designed based on the error term e . In order toemove steady-state error, we include an integrator term in the slidingurface and it is given by

= 𝑒 + 𝐾 𝑖 ∫ 𝑒𝑑𝑡 (9)

The sliding mode control is described in Appendix A . It is designedo maintain stability under high-frequency tracking. The sliding surfaceains were determined to be 14 and 12 for the longitudinal, and rotaryoFs respectively. The value of ɛ was set at 2. Integrator gains werehosen to be 8 and 4. The device characterization results with the closed-oop controller are discussed next.

123

.1. Transparency

An experiment was setup to demonstrate the transparency achievedy the closed-loop controller. A force sensor (ATI Nano 17) was attachedetween the carriage and the endoscope. The force sensor was inte-rated into the existing hardware to record position and force signalsith the same time step. In this experiment, forces and displacementsere recorded when the user pulls and pushes the endoscope in free

pace. Fig. 11 (a) and (b) show the results for the transparency exper-ment for the two cases, namely open loop and closed loop with com-ensation. It can be seen in the figure that the interaction forces in freepace, which are in the order of 3.6 N fall below 0.5 N after compensa-ion. This is because the controller measures the position and velocity ofhe device and compensates for the dynamics of the system through feed-orward terms. Furthermore, uncertainties are cancelled by the closedoop sliding-surface controller. It is to be noted that the dynamics of theser’s hand is difficult to model and is not compensated in the controller.nspite of that we see that the controller is able to compensate for allhe unmodeled dynamics such as the user hand, the fixture for the forceensor, and other uncertainties.

.2. Tracking performance

Fig. 12 shows the tracking performance of the longitudinal DoF for haptic exploration with the endoscope. In this case, the device simu-ated a virtual spring with a stiffness of 60 N/m. Fig. 12 (a) shows thepen loop controller where the desired and measured forces are com-ared. The corresponding linear velocity is shown in Fig. 12 (b). Theormalized root-mean-square error (NRMSE) in the open loop system isbout 45%. The neglected dynamics of the system in the open loop con-roller causes the tracking to deteriorate at high-frequency operation.urthermore, the open loop controller becomes unstable while render-ng high stiffness and inertia. As shown in Fig. 12 (c), the closed loopontroller with dynamic compensation discussed in Section 4 tracks theesired force even under rapid velocity changes ( Fig. 12 (d)). With thisontroller, the NRMSE falls to about 5%. Tracking performance for theotational DoF is shown in Fig. 13 . The experimental result shown inig. 12 (c) was used for estimating the time constant. Here, the user isart of the closed-loop dynamical system. The time constant for the usern the loop simulations is less than 1.5 ms. However, this might changeepending on the users grip and the boundary conditions [36] .

Force fidelity introduced by Morrell and Salisbury [26] gives a quan-itative method to compare the input and the output force signals. Fi-elity is defined as follows.

idelity = 1 −

Var ( Desired Force − Measured Force ) Var ( Desired Force )

(10)

The fidelity value of the haptic device in the longitudinal directionalculated according to Eq. (11) is 0.96. This implies that the developedaptic device is a high-fidelity device where the force distortion is kept

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 12. Tracking performance of the longitudinal DoF. (a and b) Tracking and velocity of the longitudinal DoF in open loop. (c and d) Tracking and velocity of the longitudinal DoF in the closed-loop controller.

124

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 13. Closed-loop tracking performance of the rotational DoF.

Table 4

Bandwidth of the haptic device.

End condition Longitudinal Rotational

Stiff end 41 Hz 46 Hz 0.3 m insertion 29 Hz 16 Hz 0.7 m insertion 19 Hz 14 Hz

b

l

v

1

1

4

d

o

t

a

s

t

T

A

F

w

w

s

T

I

f

o

t

1

r

s

5

c

t

o

r

i

𝐊

w

F

i

⎛⎜⎜⎝w

a

l

i

t

w

c

p

D

r

m

p

E

L

m

e

i

t

b[

{

u[

a

𝚲

t

F

a

m

g

o

p

l

t

t

d

d

elow 4%. The fidelity value in the rotational direction is 0.98. Closed-oop tracking shown in Figs. 12 and 13 is used to calculate the fidelityalues for all the DoFs. It should be noted that the signals in Figs. 12 and3 have bandwidth limited by the user input frequency, which is below0 Hz.

.3. Frequency response

Frequency response analysis is carried out on the endoscopy hapticevice to determine the closed-loop bandwidth of the system. Becausef the design structure of the haptic device, the bandwidth changes ashe inserted length of the endoscope changes. Taking this into account,part from stiff-end condition [37] , frequency response was also mea-ured at two different insertion lengths (0.3 m and 0.7 m). To simulatehe stiff end condition, a stiff- metal rod was attached to the carriage.he other end of the stiff rod was fixed and constrained from all DoFs.TI force sensor is attached at the point of the fixture to measure forces.requency response was measured by commanding periodic sine wavesith varying frequencies (1–30 Hz). The magnitude of the input forceas 5 N and the input torque had a magnitude of 50 mN.m. Fig. 14 (a–f)

hows the frequency response in translational and rotational directions.he bandwidth for the three different cases is summarized in Table 4 .t is to be noted that the bandwidths are in excess of typical motionrequencies in endoscopy. Furthermore, in all the experiments, the res-nant peaks for the longitudinal DoF are seen only beyond 10 Hz. Ashe maximum operational bandwidth of the human operator is less than0 Hz, the operation is free from resonance. The rotational DoF showsesonant peaks that are below 10 Hz. This is due to the reduced torsionaltiffness of the endoscope.

. Virtual reality interface

The virtual reality interface consists of visualization and force-omputation modules. Our physics-based force-computation solves elas-icity equations to compute the reaction forces and global deformation

125

f the upper GI tract using finite element analysis (FEA). For a mesh rep-esentation shown in Fig. 15 , the linearized static equilibrium equations given by

𝐔 = 𝐅 (11)

here K is the tangent stiffness matrix, U the displacement vector, and is the external force vector. This equilibrium system can be expandedn a block form as

𝐊 𝟏𝟏 𝐊 𝟏𝟐 𝐊 𝟏𝟑 𝐊 𝟐𝟏 𝐊 𝟐𝟐 𝐊 𝟐𝟑 𝐊 𝟑𝟏 𝐊 𝟑𝟐 𝐊 𝟑𝟑

⎞ ⎟ ⎟ ⎠ ⎧ ⎪ ⎨ ⎪ ⎩ 𝐔 𝟏 𝐔 𝟐 𝐔 𝟑

⎫ ⎪ ⎬ ⎪ ⎭ =

⎧ ⎪ ⎨ ⎪ ⎩ 𝐅 𝟏 𝐅 𝟐 𝐅 𝟑

⎫ ⎪ ⎬ ⎪ ⎭ (12)

here subscripts 1, 2 and 3 correspond to haptics DoF, unknown DoF,nd fixed DoF, respectively. The haptic DoFs are detected using a col-ision detection algorithm. They correspond to nodes that are interact-ng with the haptic interface. Fixed DoF correspond to specified nodeshat impose Dirichlet boundary conditions. All the other DoFs associatedith the free nodes termed unknown DoFs are solved using Eq. (13) . It

an be observed from Eq. (13) that the displacements cannot be com-uted by taking the inverse because the reaction forces at the hapticoFs are unknown. Furthermore, solving this complete system at haptic

ates is nontrivial. Consequently, we use a method based on Lagrangeultipliers [ 38 , 39 ]. We cast the discretized problem as a minimizationroblem as follows.

xteremize 𝐔 𝑟 , 𝚲

1 2 𝐔

𝐓 𝑟 𝐊 𝑟 𝐔 𝑟 − 𝐔

𝐓 𝑟 𝐅 𝑟 + 𝚲𝐓 (𝐂 𝐔 𝑟 − 𝐔 1

)= 𝐋 (13)

is minimized w.r.t. U r and maximized w.r.t. 𝚲. where 𝚲 is the Lagrangeultiplier vector and C is a matrix with vectors consisting of ones in the

ntries corresponding to the haptic DoF and zeroes elsewhere. Eq. (14) isn the reduced form, where K r , U r and F r are obtained after eliminatinghe fixed DoFs. The necessary condition for the problem in (11) are giveny

𝐊 𝑟 𝐂

𝐓

𝐂 𝟎

] {

𝐔 𝑟

𝚲

}

=

{

𝐅 𝑟 𝐔 1

}

(14)

By taking inverse to solve for unknowns, we get

𝐔 𝑟

𝚲

}

=

[ 𝐊 𝑟 𝐂

𝐓

𝐂 𝟎

] −1 {

𝐅 𝑟 𝐔 1

}

(15)

The inverse of the partitioned matrix in (14) can be easily computedsing the rule for partitioned matrix inverse [40] :

𝐊 𝑟 𝐂

𝐓

𝐂 𝟎

] −1 =

[

𝐊

−1 𝑟

+ 𝐊

−1 𝑟 𝐂

𝐓 (𝐂 𝐊

−1 𝑟 𝐂

𝐓 )−1 𝐂𝐊

−1 𝑟

𝐊

−1 𝑟 𝐂

𝐓 (𝐂 𝐊

−1 𝑟 𝐂

𝐓 )−1 −

(𝐂 𝐊

−1 𝑟 𝐂

𝐓 )−1 𝐂𝐊

−1 𝑟

(𝐂 𝐊

−1 𝑟 𝐂

𝐓 )−1 ]

(16)

The Lagrange multipliers are the reaction forces and are computeds

= 𝐅 1 =

[𝐂𝐊

−1 𝑟 𝐂

𝐓 ]−1 𝐔 1 (17)

It can be observed from (18) that, if the inverse of the stiffness ma-rix is available, then the reaction forces can be computed in real-time.or any given geometry and its mesh representation, a stiffness matrix,nd its inverse can be computed using FEA. Calculation of the stiffnessatrix and its inverse involves much computation. However, for any

iven mesh representation, these computations have to be done onlynce and can be performed a priori . Once the reaction forces are com-uted in real-time, the information is made available to the graphicsoop for computation of global deformation.

The model described here provides a fast method for computing in-eraction forces and global deformation at haptic rates. It may be notedhat the linearized stiffness matrix used in computation is valid for smalleformations. For large deformations, the method results in erroneouseformations. Large deformation analysis involves solving equilibrium

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 14. Magnitude (a–c), and frequency (d–f) response for stiff end condition, 0.3 m insertion length, and 0.7 m insertion length respectively.

Fig. 15. 3D model of the upper GI tract.

Fig. 16. Fabricated simulation endoscope.

e

T

t

t

5

u

w

i

e

a

o

h

F

126

quation in which the stiffness matrix is a function of displacements.he solution to this problem is computationally expensive and difficulto achieve at haptic rates. However, very large deformations that distorthe basic geometry of the GI-tract are not seen in endoscopy.

.1. Integration and VR interface

The simulation endoscope is an important part of the endoscopy sim-lator. It is designed to simulate all the features of a real endoscopehile maintaining the same ergonomics as the real endoscope. The flex-

ble tube of the simulation endoscope is same as the Olympus (GIF 130)ndoscope. The control handle of the simulation endoscope is 3D printednd its embodiment is like Olympus endoscope. Fig. 16 shows the devel-ped control handle for the simulation endoscope. The control handleouses optical encoders (HEDS-9000, 500 CPR) to track knob rotations.urthermore, custom switches are included in control handle to simulate

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 17. Snap-shot of the upper-GI simulation.

Fig. 18. Typical force characteristics for exploration of the GI model.

a

t

s

g

c

s

s

d

(

d

s

e

t

c

c

G

t

s

t

i

t

s

o

t

c

m

w

c

v

s

a

l

t

t

r

c

e

p

i

t

b

A

i

f

t

a

t

l

f

r

o

s

u

t

t

1

ir-water channels. During the development, based on the experimen-ation with gastroenterologists we found that the inertia, friction, andtiffness in the knobs of the control handle play an important part iniving the realistic feel. To passively simulate similar response in theontrol handle, we included springs, friction elements, and drives in theimulation endoscope. Springs with stiffness of 1.23 N/mm are used toimulate stiffness so that knobs turn back upon release as in a real en-oscope. O-Rings on shaft provide friction while transmission assembly2:1) with a timing belt and heavy pullies weighing 200 g simulate theynamics seen in real endoscope.

During an endoscopy simulation, the simulation endoscope is in-erted into the haptic device. Its longitudinal position is tracked by thencoder attached to the longitudinal DoF actuator. The orientation ofhe endoscope tip is sensed by the instrumented encoder knobs. Theombination of longitudinal position and endoscope tip orientation isontinuously monitored to get the endoscope position relative to theI-tract. These positions and other sensor inputs are tracked by the con-

rol unit. All the sensor information is split into 15 bytes of data and

127

ent to the VR-interface. The VR-interface updates the simulation usinghe sensed position. In our model, the complete endoscope is simulatedn real-time. Multiple collisions on the endoscope are sensed and reac-ion forces are fed to the endoscope model. The forces from the endo-cope model are sent back to the control unit using the serial port. Eachf the double decimal values from the sensed position is split into in-egers for reducing the data bytes for communication. The splitting isarefully done to avoid data loss and to capture up to the eighth deci-al place. The serial port is set at 115,000 baud rate for fast read andrite. An asynchronous read-write is implemented in the VR-interface to

ommunicate with the control unit running at 1 kHz speed. To keep theisualization detailed and still be able to simulate at haptic rates, twoeparate mesh models are used. A mapping architecture between visu-lization mesh and simulation mesh was developed for real-time simu-ation. The simulation mesh consists of around five thousand nodes andhe visualization mesh consists of 200,000 nodes. The visualization andhe simulation threads are separated to facilitate updating at differentates.

The software interface consists of a basic User Interface (UI) to fa-ilitate endoscopy training. The user can navigate to various upper GIndoscopy cases from the initial UI. Once the user selects the desired up-er GI training mode, the VR- interface together with the haptic devices launched. Fig. 17 shows a snapshot of the developed VR interface forhe upper GI endoscopy simulation. The developed VR-interface displaysoth the internal view and the global view of the endoscopy simulation.s shown in the figure, a link-view is displayed on the left side and the

nternal view is shown on the right side. The graphical visualization haseatures such as specular highlights, z-line, peristalsis motion, insuffla-ion, and red-out [41] . The texture for the simulation is generated byrtistically editing real endoscopy images. The model consists of impor-ant landmarks that are used for navigation. Some of these importantandmarks are annotated in Fig. 15 . In the current set-up, the VR- inter-ace runs on Intel PC with Nvidia GPU unit. The GPU is used only forendering and not for physics computation. As discussed in Section 5 ,ur force model is precomputation based and requires the inverse oftiffness matrix to be available at runtime. The current simulation meshses less than 2 GB of RAM for storing the inverse of the stiffness ma-rix. With this simulation set-up and an i7 PC with 12 GB RAM, simula-ion rates of more than 3 kHz are achieved. Visualization updates are at00 Hz, which is more than the required 30/60 Hz.

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Fig. 19. Realism response from subjects.

i

s

e

4

t

a

c

m

6

o

i

v

s

t

o

t

s

d

A

e

a

s

w

j

i

m

e

j

t

s

g

a

r

a

i

s

6

t

i

i

u

a

w

t

p

s

l

h

e

t

c

T

d

p

H

d

g

s

d

d

f

o

Fig. 18 shows force for two explorations of the upper GI model shownn Fig. 15 using the endoscopy simulator. In these experiments, the endo-cope was used to repeatedly probe the wall of the stomach. For the twoxperiments shown in Fig. 18 , Young’s moduli were set to be 1.9 kPa and kPa while Poisson’s ratio was set at 0.3 [42] . It can be observed fromhe figure that the endoscopy simulation system can render the forcesccurately. The entire system consisting of force reflecting mechanism,ontroller, and VR interface works seamlessly to provide complete im-ersive simulation, as explained next.

. Immersion studies

An immersion study was carried out to validate the current prototypef the endoscopy simulator. Another purpose of the study was to identifymportant areas for improvement. Formative evaluation techniques in-olve comparative studies for simulation-based training [43] . However,ince the development of the endoscopy simulator was guided by gas-roenterologists, we used experts in a specialty hospital for evaluationf the simulator. Except for [3] , there is no available literature on quan-ifying realism or establishing baseline criteria for realism in endoscopyimulators. Through the available literature and inputs from teachingoctors, we identified important parameters for assessment of realism.ccordingly, an immersion questionnaire was prepared to include differ-nt categories for evaluating physical arrangement, anatomy, graphics,nd haptic responses. Table 5 shows the questions used in the immersiontudy carried out at the Asian Institute of Gastroenterology, Hyderabad,hich is one of the largest gastroenterology centers in Asia. Nine sub-

ects consisting of seven experts and two trainee doctors participatedn the study. The experts were endoscopy surgeons with experience ofore than 10,000 endoscopy sessions. The trainees were less experi-

nced and had completed less than 100 endoscopy sessions. The sub-ects were asked to complete an upper-GI simulation from the entry tohe second part of the duodenum. In order to maintain consistency, theame upper-GI model was used for all the subjects. The subjects wereiven enough time to try out each portion of the simulation and were

128

lso allowed to re-do the simulation. Subjects were instructed to rate theealism of the simulator on a scale of 1–10, with 1 being ‘not realistic’nd 10 being ‘most realistic’. During the immersion study, a developern the team was always with the subject and noted the comments anduggestions made by the subjects.

.1. Results

The subjects were consistent with one another and the reliability ofhe questionnaire was established with Cronbach’s alpha. The standard-zed Cronbach’s alpha value for the study is 0.94 indicating an excellentnternal consistency. As shown in Fig. 19 , the overall rating for the sim-lator was good. 82% of the subjects rated the realism of the simulators “agree ” or “strongly agree ”. More than 90% of the subjects agreedith the overall haptics response. The participants were most critical of

he physical arrangement. Nearly 40% of the participants wanted im-rovements in aesthetics and the overall physical arrangement of theimulator. Mean and the standard deviation for all the questions areisted in Table 5 . The experts rated the anatomy, visual graphics, andaptic response to be satisfactory. The anatomical model had the high-st rating (mean 7.8) indicating that the model is accurate and close tohe real GI-tract. The lowest rating (mean 6.6) was given for the physi-al arrangement. The aesthetic of the system was rated low (mean 5.3).his is because the system is a functional prototype without industrialesign features. With respect to graphical rendering, the subjects ex-ressed their satisfaction for all the features and the overall simulation.owever, improvements were suggested for textures (mean 6.2).

Since haptic response is important for our simulator, we present ad-itional details on response for haptic feedback. Fig. 20 shows ratingsiven by the subjects for haptic response. The maximum number of re-ponses had a subjective rating of 8. This high rating indicates that theesign goals for the haptic device were achieved in the developed en-oscopy simulator. The overall rating haptic feedback, which includesorces along the longitudinal, rotary, and radial direction had a meanf 7.3. The experts noted that the torque feedback needed improvement

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

Table 5

Questionnaire for immersion study of the endoscopy simulator.

Sl. No. Question Mean 𝜎

Physical arrangement

1 Position of the haptic interface, screen, etc. 6.7 7.3 1.7 1.8 2 Ergonomics of the simulator 7.3 1.5 3 Aesthetics of the simulator 5.3 1.3

Anatomy 4 Geometric proportions of the upper-GI model 7.8 8.2 1 0.6 5 Position of important landmarks 7.4 1.1

Visual/graphics 6 Textures at different section of the upper-GI model 6.6 6.2 1.5 1.8 7 Features of important landmarks 7 1.5 8 Dynamic effects such as peristalsis, insufflation, and suction. 6.6 1.3

Haptic feedback 9 Force in the longitudinal DoF 7.3 7.8 1.1 0.6 10 Torque in the rotary DoF 6.3 1.2 11 Forces during insertion through the esophageal sphincter 7.7 0.8

Table 6

Desired and achieved characteristics of the endoscopy simulator.

Feature Desired specification Achieved specification

Physical characteristics Workspace Insertion- > 0.8 m Longitudinal- 1 m

Rotational- Unlimited Rotational- Unlimited rotation Weight Light weight to support portability 9 kg Footprint Small to support portability 1100 ×180 ×200 mm

Insertion and removal Multiple Multiple

Haptic characteristics Maximum force/torque Longitudinal- > 10 N Longitudinal- 11 N Rotational- Na Rotational- 255 mN.m

Sensing resolution Longitudinal- Na Longitudinal- 0.047 mm

Rotational- Na Rotational- 0.09°Fidelity High fidelity ( > 0.9) 0.96 - 0.98 Bandwidth > 10 Hz > 40 Hz (fixed end condition)

VR- interface Model Anatomically accurate geometric model Anatomically correct Simulation type Realistic and interactive Physics based (FE simulation) Simulation speed Haptic- > 1 kHz, Visualization- 30 Hz Haptic- > 3 kHz, Visualization-

> 100 Hz

Fig. 20. Frequency histogram showing user rating for haptic response for the developed simulator.

a

p

l

c

o

(

7

g

d

s

s

i

r

f

f

c

i

d

a

i

t

fi

i

A

f

(

b

a

d

P

w

nd its rating was significantly low (mean subjective rating 6.3). Thereceding result from the immersion study can be summarized as fol-ows: (i) The realism questionnaire is reliable and similarity ratings areonsistent across the subject pool; (ii) The anatomical model and theverall simulation with the haptic response is found to be satisfactory.iii) Improvements in aesthetics and textures are suggested.

129

. Summary

In this work, we described the design and development of an inte-rated system for training in endoscopy. A decoupled design of a hapticevice for simulating longitudinal force and rotational torque was pre-ented. A novel feature is a snap-fit mechanism that enables repeated in-ertion and removal of the endoscope during simulation as naturally asn the real scenario. Mechanical effects such as friction and inertia wereeduced through simple and reliable design. Dynamics-compensatedeed-forward control was developed to simulate the distinct sensation ofree space and constrained motion. A VR-interface with real-time forceomputation was developed using an FEA model. The entire system isntegrated to provide realistic and immersive simulation of upper-GI en-oscopy. The developed system meets all the design requirements and comparison table for desired and achieved specification is presentedn Table 6 . In order to provide face validation, the current prototype ofhe endoscopy simulator was also evaluated through user studies. Thendings from the immersion study suggest that the overall development

s in the right direction.

cknowledgment

This work was financially supported by the Robert Bosch Centreor Cyber-Physical Systems (RBCCPS) at the Indian Institute of ScienceIISc), Bengaluru. Authors thank Dr. Nageshwar Reddy, Dr. Pradeep Re-ala, and Dr. G. V. Rao of Asian Institute of Gastroenterology, Hyder-bad, and Dr. Lakshman K from Bengaluru for clinical guidance, helpfuliscussion, and immersion studies. The authors would also like to thankrof. Vijay Natarajan and, Dr. Nithin Shivashankar at IISc for their helpith the visualization aspects.

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

A

t

p

𝜙

𝜙

𝜙

w

m

f

𝑠

𝑠

b

𝜙

E

t

a

a

m

p

𝑉

w

𝑉

𝑉

T

𝜏

b

w

𝛽

si

t

R

[

[

[

[[[

[

[

[

[

[

[

[

[

[

[

[

[

[[[[

[

[

[

[[

[[

[

[

[

[

[

ppendix A

Consider the differential equation in (5) , which is the typical equa-ion for all the DoFs. This second order differential equation can be ex-ressed in state-space form by substituting state variables 𝜙1 and 𝜙2 forand �� respectively. The equation in state space form is

1 = 𝜙2 (18)

2 = 𝑓 (𝜙1 , 𝜙2

)+ 𝜏𝑚 (19)

here f ( 𝜙1 , 𝜙2 ) contains the b e , 𝜏e , and m e terms, and 𝜏m

now includes e term. Sliding mode control works on the principle of sliding mani-

old,

= 𝑘 1 𝜙1 + 𝜙2 = 0 , (20)

= 𝑘 1 ��1 + ��2 . (21)

Once the states are on the sliding manifold, the motion is governedy;

1 = − 𝑘 1 𝜙1 (22)

Eq. (22) is obtained by substituting (18) in (20) . As can be seen inq. (22) , when k 1 > 0 the states are guaranteed to decay exponentiallyo the origin. The rate of decay is controlled by k 1 . Therefore, the statesre guaranteed to reach the origin once they are on the manifold. Now, control law has to be designed to get the states onto the manifold andaintain them there. To do this, we define a Lyapunov function that isositive definite and continuously differentiable:

=

1 2 𝑠 2 (23)

By taking derivative with respect to time and substituting from (21) ,e get

= 𝑠 𝑠 = 𝑠 ( 𝑘 1 ��1 + ��2 ) (24)

= 𝑠 ( 𝑘 1 𝜙2 + 𝑓

(𝜙1 , 𝜙2

)+ 𝜏𝑚 ) (25)

Eq. (25) is dependent on state variables and the motor torque 𝜏m

.he sliding mode control law used in this work is given by

𝑚 = − 𝛽 sgn (s) (26)

The state variables of the system can be driven to the sliding manifoldy choosing the torque provided by the motor to be greater than 𝛽,here

> |𝑘 1 𝜙2 + 𝑓 (𝜙1 , 𝜙2

)| (27)

This guarantees that Eq. (25) is negative definite. Casting the slidingurface in terms of error and using values from Table 3 , the value of 𝛽s determined to be more than 3.4 for the longitudinal DoF and 1.2 forhe rotary DoF.

eferences

[1] Hart R , Classen M . Complications of diagnostic gastrointestinal endoscopy. En-doscopy 1990;22:229–33 .

[2] Greenwald D , Cohen J . Evolution of endoscopy simulators and their application.Gastrointest Endos Clin N Am 2006;16:389–406 .

[3] Hill A , Horswill MS , Plooy AM , Watson MO , Karamatic R , Basit TA , Wallis GM ,Riek S , Burgess-Limerick R , Hewett DG . Assessing the realism of colonoscopy simu-lation: the development of an instrument and systematic comparison of 4 simulators.Gastrointest Endos 2012;75:631–40 e633 .

[4] Desilets DJ , Banerjee S , Barth BA , Kaul V , Kethu SR , Pedrosa MC , Pfau PR ,Tokar JL , Varadarajulu S , Wang A . Endoscopic simulators. Gastrointest Endosc2011;73:861–7 .

[5] Coles TR , Meglan D , John NW . The role of haptics in medical training simulators: asurvey of the state of the art. IEEE Trans Haptics 2011;4:51–66 .

130

[6] Insko BE . Passive haptics significantly enhances virtual environments. University ofNorth Carolina at Chapel Hill; 2001 .

[7] Hannaford B , Okamura AM . Haptics, springer handbook of robotics. Springer; 2008.p. 719–39 .

[8] Ikuta K , Iritani K , Fukuyama J . Mobile virtual endoscope system with haptic and vi-sual information for non-invasive inspection training. In: Proceedings of the IEEE in-ternational conference on robotics and automation (ICRA). IEEE; 2001. p. 2037–44 .

[9] Ilic D , Moix T , Cullough NM , Duratti L , Vecerina I , Bleuler H . Real-time haptic inter-face for VR colonoscopy simulation. Stud Health Technol Inform 2005;111:208–12 .

10] Samur E , Flaction L , Bleuler H . Design and evaluation of a novel haptic interface forendoscopic simulation. IEEE Trans Haptics 2012;5:301–11 .

11] Dargar S , De S , Sankaranarayanan G . Development of a haptic interface fornatural orifice translumenal endoscopic surgery simulation. IEEE Trans Haptics2016;9:333–44 .

12] Gu Y , Kim C , Lee DY . Design of a new haptic interface for endoscopy simulation. JMed Device 2015;9:041004 .

13] EndoSim, Surgical Science ( https://surgicalscience.com/systems/endosim/ ) 14] GI Mentor, 3D Systems ( https://simbionix.com/simulators/gi-mentor/ ) 15] Korner O , Manner R . Implementation of a haptic interface for a virtual reality sim-

ulator for flexible endoscopy. In: Proceedings of the 11th symposium on haptic in-terfaces for virtual environment and teleoperator systems (HAPTICS). IEEE; 2003.p. 278–84 .

16] Woo HS , Kim WS , Ahn W , Lee DY , Yi SY . Haptic interface of the KAIST-Ewhacolonoscopy simulator II. IEEE Trans Inf Technol Biomed 2008;12:746–53 .

17] Korzeniowski P , Barrow A , Sodergren MH , Hald N , Bello F . NOViSE: a virtual naturalorifice transluminal endoscopic surgery simulator. Int J Comput Assist Radiol Surg2016;11:2303–15 .

18] EndoVR, CAE Healthcare ( https://caehealthcare.com/surgical-simulation/endovr/ ).19] Lee S-H , Park Y-K , Cho S-M , Kang J-K , Lee D-J . Technical skills and training of upper

gastrointestinal endoscopy for new beginners. World J Gastroenterol 2015;21:759 . 20] Katti A , Chakravarthy S , Ananthasuresh GK . A haptic device for entry into the throat

in endoscopy. In: Proceedings of the TrC-IFToMM symposium on theory of machinesand mechanisms, Izmir, Turkey; 2015 .

21] Carter F , Schijven M , Aggarwal R , Grantcharov T , Francis N , Hanna G , Jakimowicz J .Consensus guidelines for validation of virtual reality surgical simulators. Surg EndosOther Interv Tech 2005;19:1523–32 .

22] Appleyard MN , Mosse CA , Mills TN , Bell GD , Castillo FD , Swain CP . The measure-ment of forces exerted during colonoscopy. Gastrointes Endosc 2000;52:237–40 .

23] Shergill AK , McQuaid KR , Rempel D . Ergonomics and GI endoscopy. GastrointestEndosc 2009;70:145–53 .

24] Massie TH , Salisbury JK . The phantom haptic interface: a device for probing vir-tual objects. In: Proceedings of the ASME winter annual meeting, symposium onhaptic interfaces for virtual environment and teleoperator systems. IOS Press; 1994.p. 295–300 .

25] Tavakoli M , Patel R , Moallem M . Design issues in a haptics-based master-slave sys-tem for minimally invasive surgery. In: Proceedings of the IEEE International Con-ference on Robotics and Automation (ICRA’04). IEEE; 2004. p. 371–6 .

26] Morrell JB , Salisbury JK . Parallel-coupled micro-macro actuators. Int J Robot Res1998;17:773–91 .

27] Kilchenman R , Goldfarb M . Force saturation, system bandwidth, information trans-fer, and surface quality in haptic interfaces. In: Proceedings 2001 ICRA. IEEE inter-national conference on robotics and automation. IEEE; 2001. p. 1382–7 .

28] Ackerman MJ . The visible human project. Proc IEEE 1998;86:504–11 . 29] Ulrich KT . Product design and development. Tata McGraw-Hill Education; 2003 . 30] Nise NS . Control systems engineering. John Wiley & Sons; 2007 . 31] Longnion J , Rosen J , Sinanan M , Hannaford B . Effects of geared motor characteristics

on tactile perception of tissue stiffness. Stud Health Technol Inform 2001;81:286–92 .32] Adams RJ , Hannaford B . Stable haptic interaction with virtual environments. IEEE

Trans Robot Autom 1999;15:465–74 . 33] Samur E . Systematic evaluation methodology and performance metrics for haptic

interfaces. École Polytechnique Fédérale de Lausanne; 2010 . 34] Khan S , Andersson K , Wikander J . Dynamic based control strategy for haptic devices.

In: Proceeding IEEE world haptics conference (WHC); 2011. p. 131–6 . 35] Khalil HK , Grizzle J . Nonlinear systems. New Jersey: Prentice Hall; 1996 . 36] Hayward V , Astley OR . Performance measures for haptic interfaces, robotics re-

search. Springer; 1996. p. 195–206 . 37] Samur E . Performance metrics for haptic interfaces. Springer; 2012 . 38] Cotin S , Delingette H , Ayache N . Real-time elastic deformations of soft tissues for

surgery simulation. IEEE Trans Vis Comput Graph 1999;5:62–73 . 39] Cook RD . Concepts and applications of finite element analysis. John Wiley & Sons;

2007 . 40] Rao CR , Mitra SK . Generalized inverse of matrices and its applications. New York:

Wiley; 1971 . 41] Bansal T . Development of a graphical interface for an endoscopy simulator, computer

science and automation. Indian Institute of Science; 2015 . 42] Lim Y-J , Deo D , Singh TP , Jones DB , De S . In situ measurement and modeling of

biomechanical response of human cadaveric soft tissues for physics-based surgicalsimulation. Surg Endosc 2009;23:1298–307 .

43] Medley-Mark V , Weston CB . A comparison of student feedback obtained from threemethods of formative evaluation of instructional materials. Instr Sci 1988;17:3–27 .

S. Chakravarthy, M.V.S. Balakuntala and A.M. Rao et al. Mechatronics 56 (2018) 115–131

t

m

m

t

U

(

e

Shanthanu Chakravarthy received the Bachelor’s degreefrom the M.S. Ramaiah Institute of Technology, Bengaluru, In-dia, in 2007, the M.S. degree from the University of Utah, SaltLake City, in 2009, and a Ph.D. degree from the Indian Insti-tute of Science, Bengaluru. His research interests include hap-tics, virtual reality simulators, robotics, and compliant mech-anisms.

Mythra V.S. Balakuntala received the B.Tech. degree fromthe Indian Institute of Technology Madras, Chennai, India,in 2015. He is currently preparing for his graduate studies.Varun’s research interests include robotics, Mechanism design,and control.

Ashwin M. Rao received the Bachelor’s degree in MechanicalEngineering from the M.S. Ramaiah Institute of Technology,Bangalore, India, in 2013. He recently completed his Master’sdegree in Mechanical Engineering from the Virginia Polytech-nic Institute and State University, Blacksburg, Virginia, USA.His research interests include robotics and control.

131

Ravi Kumar Thakur received his Bachelor’s degree in Physicsfrom the National Institute of Technology, Calicut in 2015. Heis currently pursuing his Master’s degree from the Indian In-stitute of Information Technology (IIIT), Sri City. His researchinterests include controls and machine learning.

G. K. Ananthasuresh received the B.Tech. degree from the In-dian Institute of Technology Madras, Chennai, India, in 1989and the Ph.D. degree from the University of Michigan, AnnArbor, in 1994. He is currently a Professor of mechanicalengineering with the Indian Institute of Science, Bangalore,India. He worked as a Postdoctoral Research Associate withthe Massachusetts Institute of Technology, Cambridge (dur-ing 1995–1996), taught at the University of Pennsylvania,Philadelphia (during 1996–2004), and served as a Visiting Sci-entist with the University of Cambridge, Cambridge, U.K., andthe Katholike Univesiteit, Leuven, Belgium. He is on the Edi-torial Boards of eight journals. He is a coauthor of more than180 papers in journals and conferences, as well as a textbook,

wo edited books, and ten book chapters. His current research interests include compliantechanisms, kinematics, multidisciplinary design optimization, microsystems technology,icro- and mesoscale manufacturing, protein design, and cellular biomechanics. Dr. Anan-

hasuresh received the National Science Foundation CAREER Award (1998–2002) in the.S. and the Swarnajayanthi Fellowship (2007–2012) and Shanti Swarup Bhatnagar Prize

2010) in India, as well as seven best paper awards at international and national confer-nces.