1

Robotics for Rehabilitation and Medical Applications

Domenico Prattichizzo

Robotics & Systems Lab University of Sienawww.dii.unisi.it/prattichizzoprattichizzo@dii.unisi.it

Centro per la ricerca scientifica e tecnologicaITC - IRST Trento, December 2003

University of Siena

2

OutlineVisuo-Haptic Interaction

1) Grasp Rehabilitation

2) the FeTouch Project

Eye Tracking

3) ASTIDET

University of Siena

3

Haptic Interaction ?

University of Siena

4

Touch is very important in medicine

Increasing telepresence

through video-haptic interfaces

HAPTIC interfaces

from Greek haptesthai : to touch

University of Siena

5

Anthromorphic Robot Arm (kinematics)

• sensors to measure end-tip position

• actuators to apply end-tip force

University of Siena

6

• measure finger positions

• get finger position

• compare with the virtual obj. (collision detection)

• if the hand-avatar is inside, apply a force …

Haptic Interface Principle

University of Siena

7

Ungrounded Haptic Interfaces

joint angles are

measured and

forces applied

relative to mount

point on the

body

Cybergrasp by Immersion inc.

University of Siena

8

VR

Haptic Interface

Visual Interface

Visio-Haptic Interaction

bidirectional flow

University of Siena

9

OutlineVisuo-Haptic Interaction

1) Grasp Rehabilitation

2) the FeTouch Project

Eye Tracking

3) ASTIDET

University of Siena

10

Grasp Rehabilitation

University of Siena & University Vita-Salute San-Raffaele

Other partners

Service de Reeducation, Hopitaux Universitaires, Switzerland

Institut National de la Sante et de la Recherche Medicale, France

Institut Guttmann, Spain

FCS Control Systems B.V, Netherlands

University of Siena

11

Grasp rehabilitation goals

1) Rehabilitation units equipped with robotic systems helping patients with motor deficits to relearn basic manual skills such as to open doors, use forks and knives, drink a glass of water, or button a shirt.

2) Virtual Reality Environments that allow pre-surgery patients to face the expected deficits after a surgical intervention, by mimicking its effects on fine manual skills.

University of Siena

12

Robotic upper-limb rehabilitation

HapticMasterTM with the Endo-Ring (project GENTLE/S).

University of Siena

13

Upper-limb rehabilitation @ MIT

• MIT-MANUS (1990s) - The robot provided guidance, assistance or resistance while the patient executed a pointing movement.

Robot-aided patients improved more than the control patients.

The gains obtained with this technique remain significant after 3 years.

{{{{{{{{{{{{{{{{{{{{{{

Burke Hospital-MIT group

Palo Alto Veteran's Affairs Hospital / Stanford group

Spaulding Rehabilitation Hospital (1999)

Baltimore VA Hospital (2000)

University of Siena

14

Upper-limb rehabilitation in Europe

• MULOS

Motorised Upper Limb Orthotic System is a 5 dofs robot mounted on a wheel chair that can move the joints of the upper limb under joystick control.

• REHAROB

This project used industrial robots to passively manipulate the arm of patients.

• GENTLE project

…

University of Siena

15

Robotic rehabilitation @ Siena

Grasp rehabilitation

Simpler robotic devices

University of Siena

16

Robotic rehabilitation @ Siena/Milan

The Grip Analyzer. The Grip Analyzer is essentially a real object instrumented with several six degree-of-freedom (DOF) transducers for measuring contact forces.

University of Siena

17

Robotic rehabilitation @ Reachin

The Virtual Grasping Environment. Simulation of a double point contact interaction using two degrees of freedom devices.

ReachinTM setup

University of Siena

18

Robotic rehabilitation @ Siena

Multi finger haptic interaction.

Stability analysis Performance evaluationDeformable objects

Stabilizing a multi contact haptic intercation:Problem overview

University of Siena

19

References

University of Siena

D.Prattichizzo, J.K. Salisbury, and A. Bicchi, “Contact and grasp robustness measures: Analysis and experiments.’’ in Experimental Robotics IV}, Lecture Notes in Control and Information Sciences, pages 83--90. Springer-Verlag, 1997.

A.Bicchi and D.Prattichizzo, in \Analysis and optimization of tendinous actuation for biomorphically designed robotic systems\ in Robotica 18:23--31, 2000.

Workshop ``Multi-point interaction: from robotic hands to haptic interfaces'‘ organized by F. Barbagli, D. Prattichizzo and K. Salisbury at the IEEE International Conference on Robotics and Automation}, 2004, (submitted). To be published on a Springer Verlag Serie.

F. Barbagli, D. Prattichizzo, J.K. Salisbury, ``Multi-contact haptic interaction with deformable objects: amulti-rate approach'' in ISRR 2003, the 11th International Symposium of Robotics Research, October 19-22, 2003, Siena, Italy. To be published on a Springer Verlag Serie.

20

OutlineVisuo-Haptic Interaction

1) Grasp Rehabilitation

2) the FeTouch Project

Eye Tracking

3) ASTIDET

University of Siena

University of Siena

The FeTouch project @ Siena

Robotics & Systems Lab

Dip. di Pediatria, Ostetricia e Medicina della Riproduzione

University of Siena

www.fetouch.org

22

• • training & simulation,training & simulation, for:

– diagnosis

– tissue palpation

• Minimally Invasive SurgeryMinimally Invasive Surgery (MIS)

– training through simulation

– the real thing: “fly by wire” to improve interaction

• telesurgerytelesurgery

demonstrated on a human in 2002

(at the EITS in Strasburg from New York)

Medical Applications

University of Siena

23

Siemens Sonoline Elegra Dipartimento di Pediatria, Ostetricia e Medicina della Riproduzione.

Isosurface

Vision (25fps)

Haptic Interface (1KHz)(tactile rendering)

Confluence of Ultrasound and Haptic Interfaces

real patient data

University of Siena

24

University of Siena

The FeTouch System

25

Direct Volume Rendering(Ray-Casting)

Isosuphase Extraction(Marching Cubes)

Volume and Surface Rendering

Marching Cubes (triangles)

NEW !

(geometric)

University of Siena

26

Ultrasound Image Segmentation

Segmentation and Surface Extraction

Marching Cubes (triangles)

University of Siena

27

KHz1

F RF

Haptic Principle1. [Avatar] Measure hand position (encoders)

2. [Collision Detection] Localize the avatar in VR

3. [Force Rendering] Apply force to the operator(motor)

4. [Visual Feedback] Modify VR (deformable objects)

Problems

fast force feedback loop

slow deformable object dynamics …

Internal organ modeling

University of Siena

28

Proxy and force rendering

University of Siena

29

US3D:US3D:

US3DTouch:US3DTouch:

Fetus face (baby.vtk) VIDEO

Heartbeat

US3D & US3DTouch Session

to know more

www.dii.unisi.it/fetouch

www.fetouch.org

University of Siena

30

Neonatology

in cooperation with

University of Siena

31

References

University of Siena

B. La Torre, D. Prattichizzo, F. Barbagli, and A. Vicino.``The fetouch project, a tool for augmenting the fetus perception.\ in The Inter. Journal of Medical Robotics and Compurer Assisted Surgery, Jan.2004.

B.~la~Torre, D.~Prattichizzo, F.~Barbagli, and A.~Vicino. “The fetouch project.\ in Proc. IEEE Int. Conf. on Robotics and Automation, 2003.

B. La Torre, D. Prattichizzo, F. Barbagli, and A.Vicino, ``The fetouch system: Visual-haptic rendering of fetuses.\ in Proc. Euro{Haptics 2003, Dublin, Ireland, 6-9 July 2003.

32

OutlineVisuo-Haptic Interaction

1) Grasp Rehabilitation

2) the FeTouch Project

Eye Tracking

3) ASTIDET

University of Siena

University of Siena

Robotics & Systems Lab

Dep. of Neurological and Behavioural Sciences

University of Siena

ASTIDET, Advanced STImuli Design for Eye Tracking Easy Applicability for Ocular Movement Study in Neurology

34

PC

Eye movement data

Eye Tracking

University of Siena

35

Eye MovementsEye Movements Pan/Tilt Infrared camera (ASL)Pan/Tilt Infrared camera (ASL)

observed by

controlled by

Control Unit Control Unit Pupil, CR, Line of gazePupil, CR, Line of gaze

Eye-Tracking: Setup

University of Siena

36

•CCD camera-IR emitting diodes with light source coaxial with optical axis

•Detection of light reflections from cornea (CR) and bright pupil.

•Real time processing of eye image sequences and extraction of pupil and CR centroids.

•Line of gaze computed using pupil and CR centroids.

Infrared Video-Based Eye-Tracker

University of Siena

37

ASTIDET ASTIDET ©©

Stimulus generator

University of Siena

38

Experimental Results: Saccades Experimental Results: Saccades StudyStudy

Tg=20cs Ta=180cs

12 °

University of Siena

39horizontal movement

vertical movem

entUniversity of Siena

40

Eye tracking for medical diagnosis

• Disease causes abnormal oscillations of the eyes

• Images of stationary objects move excessively on the retina (blurring of vision and oscillopsia)

University of Siena

41

Ocular rehabilitation

Generated scene can change with respect to the subject’s eye movements!

University of Siena

42

Scene Viewing-FixationScene Viewing-Fixation

43

References

University of Siena

A. Rufa, D. Prattichizzo, G.L. Mariottini, A. Vicino, and A. Federico, ``Advanced STImuli Design for Eye Tracking - ASTIDET: a software tool for scene generation, data acquisition and analysis in ocular movements study,'‘ in Proc. of the American Academy of Neurology Conference, San Francisco, USA, April 2004,

A. Rufa, D. Prattichizzo, G.L Mariottini, A. Patrichi, A. Vicino, and A. Federico, ``Advanced STImuli Design for Eye Tracking (ASTIDET): easy applicability for ocular movement study in neurology,'' in Atti del Simposio Italiano di Neurologia, Rome, Italy, October 2003.

44

Concluding

Visuo-Haptic Interaction

Grasp Rehabilitation

the FeTouch Project

Eye Tracking

ASTIDET

Task Driven Robotic Devices

Towards Simplicity (home tools)

Modularity

rslab.dii.unisi.it

www.dii.unisi.it/prattichizzo

University of Siena