Virtual surgery simulatorsSimulatori per chirurgia virtualeElena De Momi, PhD, Politecnico di Milano, Department of Electronics, Information and Bioengineering

Alessandro Perin, PhD, Neurosurgery Department, Fondazione IRCCS Istituto Neurologico Nazionale "C. Besta", Milan

XXXIV Scuola Nazionale di Bioingegneria, 21- 24 Settembre 2015, Bressanone

Elena De Momi

From the cockpit to the ORThe performance combined

2

FatigueWorkload

Time pressure

TechnologyLives at risk

Responsibilities

Team management

Stress

Risk management

Elena De Momi

Situation awareness modelEndsley model

3

Endsley, Toward a Theory of Situation Awareness in Dynamic Systems, 1995

Elena De Momi

How can they improve their performances? 4

Plan

PreparePractice

Elena De Momi

The need of simulators 5

Simulation of the vehicle behavior

Simulation of environment

Emergency situations handling

Simulation of emergency situations

Team decision making training

Pilot performances (to maintain flight privileges) checking

Elena De Momi

How to make surgery more effective? 6

Training is the perfect answer to this need

Patients’ outcome Pathology social costs

Elena De Momi

Surgical training 7

Tests on human cadavers

• PROS• Equal morphology

• CONS• Costs• Ethical issues• Changes in tissues

behavior• Different

physiology

Tests on live animals

• PROS• Similarities in

tissue behavior• CONS

• Different morphology

• Ethical issues• Costs

Tests on phantoms

• PROS• Reproducibility

(objective evaluation)

• Similar morphology• CONS

• If the models are detailed they can be expensive

• Wrong tissue behavior

Elena De Momi

Surgical simulators ADVANTAGES 8

- Simulate the entire intervention workflow or particular step/actions

- Take team decision – discuss

- Simulate emergency situations/ risks management

- Check procedure

- Assess skills

Preparation of launch

Orbit injection

Orbital flight

Elena De Momi

How can we build simulators? 9

"There is no particular mystery in animation...it's really very simple, and like anything that is simple, it is about the hardest thing in the world to do. "Bill Tytla at the Walt Disney Studio, June 28, 1937

2D pictures

Computer animation: deformable models in a simulated physical world

first feature-length computer animated film1995

Adding rules and laws:

Terzopulos, Artificial fishes, 1994

Elena De Momi

The simulator components 10

From medical

images to models

• Segmentation• Surface meshes

Volumetric models

•Constitutive laws•Volumetric meshes

Interaction

•Haptic interfaces•Interaction visualization•Interaction force rendering

Elena De Momi

From medical images to surface models 11

Image dataset

Elena De Momi

Marching squares 12

Wikipedia

Lorensen WE, Cline HE, Marching Cubes: A high resolution 3D surface construction algorithm. In: Computer Graphics, 1987

Elena De Momi

From medical images to surface models 13

Elena De Momi

From images to volumetric models 14

Elena De Momi

The simulator components 15

From medical

images to models

• Segmentation• Surface meshes

Volumetric models

•Constitutive laws•Volumetric meshes

Interaction

•Haptic interfaces•Interaction visualization•Interaction force rendering

Elena De Momi

Tissue models 16

3

3

2

3

1

3

3

2

2

2

1

2

3

1

2

1

1

1

Xx

Xx

Xx

Xx

Xx

Xx

Xx

Xx

Xx

XxF

22

2

1

1212

J

WJWT

Soft tissues have non-linear behavior: they are non-homogeneous and anisotropic

F deformation gradientx current position of the pointX point in the reference configuration

T uniaxial stress tensorW deformation energyλ eigenvalue of FJ deformation invariants

Holzapfel, Nonlinear Solid Mechanics: A Continuum Approach for Engineering, Wiley, NY. 2004

Elena De Momi

Soft tissue models 17

32exp12 22212

BBBWT

13exp 121 JBBW dτ3dτd2

0 3212

ttW

WT

Liver model

W deformation energyT uniaxial stress tensor

Brain model

B1, B2 material parameter µ shear modulusα experimentally determined

Roan E, The Nonlinear Material Properties of Liver Tissue Determined From No-Slip Uniaxial Compression Experiments. ASME. J Biomech Eng. 2006

Miller, Mechanical properties of brain tissue in tension. 2002

Elena De Momi

Interaction models: Finite Element Models 18

i-th element

Discretized geometry

Nodal degrees of freedom in x, y, z directions

Starting from nodal displacement, stress and strain are estimated in every point (Shape function)

j-th node

constraint

SOLUTIONDisplacements of the nodes estimation

DEFORMATIONINITIAL CONFIGURATION

FORCES applied on nodes

Elena De Momi

Interaction models: Finite Element Models 19

eee uKf

fne nodal forcesune elements node displacementK element stiffness matrix

x cartesian coordinates of a point on an elementN shape function matrixu cartesian coordinates of the element nodes

x = Nu

Elena De Momi

Interaction models: Mass-spring models 20

ij

ijijijsi lkxx

xf

fi acts on spring i and it is generated by spring that connects i and j

xij is the distance vector of masses in xiand xj

lij is the initial length

Provot, Deformation Constraints in a Mass-Spring Model to Describe Rigid Cloth Behavior, 1995

row = 4;col = 7;stiffness = -60; % N/mdamping = -0.5; % Ns/mmass = 0.1; % Kg

Elena De Momi

The simulator components 21

From medical

images to models

• Segmentation• Surface meshes

Volumetric models

•Constitutive laws•Volumetric meshes

Interaction

•Haptic interfaces•Interaction visualization•Interaction force rendering

Elena De Momi

The interaction model 22

Displacement estimation

Force estimation

Force feedback to the user

GRAPHICAL DISPLAY

Elena De Momi

Haptic rendering 23

Elena De Momi

Force feedback enhancement

Tactile sensitivity (cutaneous sensation) depends on size, density, frequency range, nerve fiber branching and type of stimulation (skin motion or sustained pressure).

They are useful to recognize the local properties of objects such as shape, edges, embossing and recessed features, thanks to a direct measure of the intensity and direction of the contact forces.

Kinesthesia: perception of limb movement and position and perception of force.

This sensory perception originates primarily from mechanoreceptors in muscles.

Elena De Momi

Haptics 25

Elena De Momi

26Balancing between accuracy and realism

Elena De Momi

Surgical applications: laparoscopy 27

The surgery is performed through a small hole (keyhole 5-15 mm) – trochar.

Laparoscope: a long fiber optic cable system which allows viewing of the affected area

Advantages:

- Reduction of pain and hemorrhage- Reduction of recovery time

Limits:

- Limited dexterity - Limited visibility - Fulcrum effect

Elena De Momi

Surgical applications: laparoscopy training 28

There are 24 SAGES Fundamentals of Endoscopic SurgeryTM (FES) test centers across North America. This number is expected to grow annually now that the FES test, as a component of the larger Flexible Endoscopy Curriculum, is becoming a requirement for certification by the American Board of Surgery (ABS) for residents in surgery graduating in the 2017-2018 academic year.

•New LAP Mentor™ Lobectomy module trains on complicated procedure of Video-Assisted Thoracoscopic Surgery (VATS)

•GI Endoscopy – Fundamental Skills training module helps learners prepare for flexible endoscopic surgery assessment

•RobotiX Mentor™ is the only VR simulator to offer training for robotic clinical procedures

Elena De Momi

Surgical applications: robotic surgery 29

Advantages:

- Improved dexterity

- Improved intervention intuitiveness

Limits:

- Lack of force feedback

- High costs

- Long learning curve Lack of training increases risks for patient (1700 legal actions against Intuitive mainly for issues related to lack of training)

da Vinci, Intuitive Surgical, CA, USA

Elena De Momi

Surgical applications: robotic surgery training

30

da Vinci Surgical simulator, USA

Specific task training (suturing, gastric bypass, palpation)

Team work decision making

Robotic surgery specific training

Elena De Momi

Surgical applications 31

Basic skills- Visuo-motor coordination- Bimanual cooperation

Cognitive skills- Cognitive load- Context related, feed forward control

Xron, UNIVR

Elena De Momi

Simulation centers 32

ORSIRobotic surgery training(Melle, Ghent University, Belgium)

Goodman Surgical Education CenterStanford Hospital, CA, USA

Elena De Momi

Surgical applications: neurosurgery 33

Necessity of accurate planning:- surgical target - surgical access

avoiding vessels and critical structures (e.g. eloquent areas, ventricles)

Need for training on:

- Surgical workflow, surgical steps sequence

- Team coordination, scrub nurse instrument handling

- Emergency handling (e.g. seizure, vessel rupture)

- Accuracy in targeting

- Maximum forces/ tissue stiffness

Elena De Momi

Surgical applications: neurosurgery training 34

• Various scenarios• Case specific rehearsal• Haptic feedback

• Highly accurate and realistic• Excellent haptic feedback• Multiple metrics parameter registration

Neurotouch, Canada

Imm

ersi

veTo

uch,

Bos

ton,

MA,

USA

Elena De Momi

Neurosurgery simulation center @Besta 35

Neurosurgeons training

Objective resident evaluation

Elena De Momi

Surgical applications: interventional cardiology

36

Training is needed to:

- Avoid exerting excessive force on vessel (risk of rupture)

- Limit contrast medium injection

- Avoid unnecessary fluoroscopic images

- Learn procedure workflow

Surgical procedures performed on the heart by catheterization, inserting a sheath into the femoral artery and cannulating the heart under X-ray visualization

- Avoidance of the scars and pain- Avoidance of long post-operative recovery.

Elena De Momi

Surgical applications: interventional cardiology training

37

(a) VIST-Lab (Mentice AB, Sweden) provides a body mannequin, tactile and force feedback with simulated instruments, different screens for displaying simulated fluoro, or vital images, simulation modules for different arterial diseases within coronary, carotid, renal, and iliac arteries, as well as performance metrics such as fluoroscopy time and contrast volume use

(d) ANGIOMentorTM (Simbionix, OH) provides simulated training of different interventional radiology, vascular and cardiac surgery procedures using preoperative data and simulated fluoroscopywhile enabling simultaneous access through two access sites with tactile feedback

(a)

Rafii Tari et al., Current and Emerging Robot-Assisted Endovascular Catheterization Technologies: A Review 2014

Elena De Momi

Is the simulator good? 38

- Does the system present an environment resembling that, which is encountered during a medical procedure?

- Is a skill measured by the system the specific skill desired, and not a different one?

- Can the system capture the difference between expert and novices?

- To what extent does testing performance with the simulator yield the same results as other measures?

- Does performance/ training with the simulator transfer to improvements in clinical practice?

Elena De Momi

39Robotic surgery

Micron, CMU, PA

Hearthlander, CMU, PA

DaVinci, Intuitive

Rav

en, U

W, W

A

Neuromate, Renishaw mayfield

Alf-x

, Sof

ar

Mako

Rosa, Medtech

Master-slave

Autonomous/ hands on

Hands-on

Catheters

Magellan, Hansen Medical

Neu

roAr

m, I

MR

IS

Elena De Momi

The future of simulators 40

• Excellent 3D visual rendering

• Case specific pre-operative analysis

• Integrated with intra-operative navigation systems

Elena De Momi

41

Thanks for your attention www.nearlab.polimi.it