ROBOTIZED CAMERA SYSTEM FOR REAL-TIME COACHING OF ...

International Journal of Computer Assisted Radiology and Surgery, Vol. 3, No 3-4, Sept. 2008, pp. 241-248(8)

1

ROBOTIZED CAMERA SYSTEM FOR REAL-TIME COACHING OF CLINICIANS IN EMERGENCY ROOM

R. Lemieux1, F. Michaud4, P. Masson3, C. Bellemarre1, M. Martin1,2, M.-È. Bernard3, P. Fauteux3, C. Julien3, M. Lalonde-Filion3, C. Morier3, A. Morin-Guimond3, M.-A. Patry3,

A. Saint-Pierre3 1Centre hospitalier universitaire de Sherbrooke, 2Dept. Traumatology, 3Dept.

Mechanical Engineering, 4Dept. Elec. Eng. & Comp. Eng., Université de Sherbrooke, Sherbrooke (Québec) CANADA J1K 2R1

ABSTRACT Purpose: To compensate for the shortage of emergency specialists and to maintain quality of medical care in remote regions. Methods: Develop and validate the concept of real-time coaching using a robotized camera system ceiling-mounted above a stretcher. Exploit a final prototype to evaluate its impact on surgical procedures under controlled conditions. Results: Preliminary results demonstrate: i) the system allows a remote practitioner to easily view all surgical procedures used in emergency room, and ii) a specialist can coach a non-specialist to perform a surgical procedure using proper kinesiologic gestual with many individuals surrounding the stretcher. Conclusion: The system could provide help on demand and improve the level of services in trauma medicine for the population of regions where trauma surgeons are unavailable. The system could also decrease medical cost by providing remote support on surgical procedures used to stabilize unstable polytraumatized patients at their arrival in the trauma room. KEY WORDS Teletraumatology, traumatoloy, remote support, robotized cameras, telecoaching

1. Introduction The Canadian public health care system is facing a shortage of emergency specialists, including trauma surgeons [1]. Therefore, local clinicians are more than ever called to provide medical care in emergency rooms. This situation is especially true in remote regions of Canada [2] where there are no trauma surgeons or emergency specialists. Having approximately 15 minutes to perform the procedures to save the lives of the patients,

clinicians working in emergency rooms are faced with life-and-death situations. By lack of continued training opportunity, clinicians may use techniques that are not best suited to obtain optimal results. Emergency medicine speciality is a relatively new program, and providing telementoring services constitutes a very promising avenue to assist local clinicians in performing surgery procedures using proper kinesiologic gestual on unstable polytraumatized patients. Tele-assistance by an expert is a concept that is widely used in other fields of medicine such as radiology [3], pathology [4] and psychiatry [5]. Real-time teleconsultation with an emergency specialist offers significant benefits to patients in regions where no specialists are available and patients are frequently transferred to specialized centres [6, 7]. Significant reduction in medical assistance costs and in response time of professionals also resulted from teleconsultation in emergency medicine [7]. The practice of tele-emergency through videoconferencing is currently available for minor injuries [7]. Tachakra et al. [8] showed concordance above 95% in comparing telemedicine with face-to-face consultations for trauma management of minor injuries, both in terms of diagnostic precision and quality of medical assistance. In the case of major injuries, preliminary studies indicate significant benefits of telemedicine in both patient life support surveillance [9] and patient management [10]. A pilot study conducted at the Centre hospitalier universitaire de Sherbrooke (CHUS) indicated that real-time coaching of emergency physicians in performing surgical procedures on polytraumatized patients would be the best way to 1) assist emergency physicians with difficult cases, 2) help them gain manual dexterity and 3)


2

maintain their level of expertise up-to-date. Preliminary results reported by Rogers et al. [11] also conquer with this assessment by demonstrating that two cameras were best suited for such application. One camera was attached to the ceiling above the trauma stretcher, while a second camera was fixed on the top of the telemedicine workstation. The cameras’ PAN, TILT and ZOOM were controlled remotely. The trauma surgeon could appreciate the performance of the trauma team by switching between the two camera views, one providing a local view of the intervention, while the second giving a global view of the trauma room. To push this concept further, and because there are no robot-based systems for trauma medicine available on the market, a first proof-of-concept prototype was developed from 2004-2006 by the Centre hospitalier universitaire de Sherbrooke (CHUS), the Faculty of Medicine and Health Sciences (FMHS) and the Faculty of Engineering of the Université de Sherbrooke. Our objective was to come up with a system using two mobile cameras ceiling-mounted on rails above the stretcher (instead of only one) for real-time remote coaching in traumatology. A trauma surgeon remotely assists a local clinician in performing surgery procedures using proper kinesiologic gestual on unstable polytraumatized patients. More specifically, a major interest of the CHUS emergency medicine specialty is to use kinesiology movement decomposition as a tool to teach emergency physicians the proper techniques to perform eight surgical procedures on unstable polytraumatized patients (endotracheal intubations, comb tube installation, cricothyroidotomy, needle tracheotomy, thoracotomy, pericardiosynthesis, central venous approaches, and sapheno-femoral dissection). Overall, three prototypes were developed. In addition to the results reported for the first two prototypes [12, 13], this paper presents the third prototype and a qualitative assessment of the system in controlled conditions using corpses. Such assessment is a prerequisite to the system deployment in trauma rooms for real emergency cases.

2. Robotized Camera System A common feature of all prototypes developed is to have two independently controllable mobile cameras, ceiling-mounted above to a trauma stretcher. They have the capability of PAN, TILT and ZOOM and movement capacity on a predefine area depends on the mechanical design. To change viewpoints in case of visual obstruction, either one of the two cameras can provide a global view of the patient while the other focuses on the surgical procedure. Real-time communication exists between the remote trauma surgeon and the on-site emergency physician using videoconferencing CODECS that code and transmit video and audio stream (e.g., H.263 and G.722). The robot control commands that position the cameras are transferred directly through Internet using the TCP/IP transmission protocol. Audio communication between the local clinician and the remote surgeon is done through a wireless microphone mounted on a headset. This setup minimizes human background noise and conversation while avoiding physical interference with medical equipments located in the trauma room. The positions of the cameras are set through a user interface. The quality of this interface and the operator’s experience in operating the system at distance are two factors that have a direct impact on the safety and performance of such systems [14, 15]. The user interface must be designed to minimize the operator's cognitive load and to direct his attention on the intervention rather than on the tele-operating system. Figure 1 shows the overall architecture of our robotized camera system. The first prototype, shown in Figure 2, is composed of two independently controllable cameras, ceiling-mounted on rails above and perpendicular to the trauma stretcher. The remote trauma surgeon controls the cameras using two independent joysticks and a graphical user interface. Semi-autonomous control of the cameras is achieved by combining the commands given by the remote surgeon through the joysticks with preset positions derived from the known positions of the medical team around the stretcher for a specific technique. Figure 3 illustrates the user interface of the first prototype.


3

Figure 1. Architecture of robotized camera systems.

(a)

(b)

Figure 2. First prototype using two side rails. (a) The prototype in a traumatology room. (b) The prototype in a controlled environment (e.g., anatomy room at the CHUS).

(a)

(b)

Figure 3. User interface of the first prototype. (a) User interface for the remote trauma surgeon. (b) The trauma surgeon remote control station. The second prototype (Figure 4) was designed to correct the limitations observed on the first prototype, e.g. to increase the range of motion of at least one camera while not interfering with


4

equipment usually found in trauma rooms. The second prototype is composed of one camera mounted on a two-axis rail system located directly above the stretcher. It provides a wider range of viewpoints and allows close-up viewing of critical emergency techniques. A second camera is mounted on a single axis lateral rail located at the tail end and above the stretcher. It provides a global view of the patient. The cameras are controlled independently by a more sophisticated user interface, shown in Figure 5, based on the capabilities of a tactile screen that provides storage for the target locations for semi-automated repositioning of the cameras in case of obstruction.

Figure 4. The second prototype using a X-Y table and one lateral rail, in the anatomy room.

Figure 5. User interface of the second prototype. The third and final prototype (Figure 6) was designed also to improve from limitations observed with the previous prototypes, mainly to facilitate installation and usability, and to conduct a clinical assessment on real emergency cases. The third prototype is set to comply with

biomedical standards (e.g., Z32-99 Electrical Safety and Essential Electrical Systems in Health Care Facilities) and CSA standards. The concept is to fix a camera at the end of a SCARA-type robotic manipulator. Each manipulator can cover an area of 0.64 m2. Its design is optimized to allow clearance for other medical equipments installed in the emergency room. This concept provides more robustness of the mechanical components, minimizes dust accumulation and improves aesthetics. This concept is used to install two ceiling-mounted cameras above the trauma stretcher. The arms manipulators are controlled by an enhanced and more intuitive graphical interface to provide the remote specialist with better confidence in manipulating the system and more time to assist clinicians in real time. Figure 7.b illustrates the initial positions of the cameras on the user interface. Cameras are centered over the patient, maximizing the surgeon's visualization and reducing manipulation of the system when coaching is required.

(a)

(b)

Figure 6. The third prototype using two SCARA arms. (a) Schematics of one SCARA manipulator for positioning a camera. (b) The third prototype in the anatomy room.


5

(a)

(b)

Figure 7. The third prototype user interface. (a) The remote control station. (b) User interface.

3. Technical and Preliminary Clinical Test Results Teletraining and telecoaching are the two applications first validated with the robotized camera system, both in controlled conditions (e.g., anatomy room at the CHUS). Using the first prototype, cricothyroidotomy, thoracic drain, comb tube intubation, endotracheal intubation and venous access were performed on corpses. The system showed its capability to easily view these surgical procedures while performed for the first time by a non-physician tele-assisted by a trauma surgeon. Figures 8 to 10 show the capture of static images of surface injuries taken by the two cameras. Both cameras are synchronized and show the same procedure from different points of view. These trials demonstrated that a specialist can coach a non-specialist to perform a surgical procedure using a robotized-camera system, in situations where one or many individuals surround the stretcher.

Figure 8. Thoracic drain performed by a non-physician. This surgical procedure was tele-assisted by a trauma surgeon.

Figure 9. Cricothyroidotomy performed by a non-physician. This surgical procedure was also tele-assisted by a trauma surgeon.

Figure 10. Venous access performed by a physician. However, when a team of professionals surrounded the patient, it became obvious that the ergonomics of the user interface developed to position the cameras needed to be improved. This is especially true when performing techniques near the head or the neck, such as the situation shown by Figure 11. It is therefore important to evaluate the capacity of the system to handle such situations before installing it in trauma rooms.

Figure 11. Inappropriate field of view provided by the cameras using the first prototype during an endotracheal intubation.


6

We assessed the capacities of the third prototype when installed in the anatomy room at CHUS which provided a totally controlled environment. Three emergency physicians, one trauma surgeon and two paramedical technicians participated to the study. They played two roles : the local clinician that performed surgical procedures on corpses and the remote specialist that operated the system. Prior to the assessment period, all participants had 15 to 25 minutes to familiarize themselves with the system. Technical assistance was provided to make sure that they could easily operate the system. During the trials, one participant, standing near the stretcher, intentionally obstructed the cameras fields of view while a surgical procedure is performed to force the remote specialist (a second participant) to reposition the cameras in order to recover full fields of view on the surgical procedure. All participants had to escape more than six obstructions during the performance of surgical procedures. The assessment period lasted between 10 to 20 minutes per remote practitioner. Afterward, we interviewed all participants to first evaluate the system’s capacity to provide visual information necessary to properly assist participants that performed the surgical procedures (cricothyroidotomy, thoracic drain, comb tube installation, endotracheal intubation and venous access) and second to evaluate the user’s ability to operate the system while viewing and analyzing the performance of the surgical procedures. A questionnaire was prepared for this purpose. Two rooms at CHUS were used in this assessment study. First, the anatomy room, where the system was installed, simulated a trauma room. A second room, adjacent to the anatomy room, simulated the specialist office. Images captured by the cameras were transferred by videoconferencing systems, using 1472 kbps (736 kbps per camera) network bandwidth, H323 transmission protocol, h.264 video codec and G.722 audio codec. The CHUS local area network was not part of this evaluation because the assessment was performed in ideal transmission conditions where no packet lost was observed before and during the assessment. During the assessment, we insisted on the following measured variables: image quality with and without robot movements, noise level made by the system, time spent by the system to

move cameras from one position to another using four velocities, time spent by the operator to setup a new system position, precision of cameras positioning with respect to user’s intent, participant’s satisfaction in operating the users interface, system’s friendliness from the user’s point of view. Image quality during robot movements was considered as good while it was considered as very good in the absent of robot movements. All participants were capable of viewing perfectly 14 gauge needles penetrating the skin during the venous access procedure, while cameras were located at 1.2 m away from the stretcher. However, when network bandwidth was reduced to 768 kbps (or 384 kbps by camera) for video coding, image quality was judged as poor and considered not useable for tele-assistance in traumatology. Noise level to velocity ratio was minimized when system’s elbow and shoulder rotated at velocities of 90 000 pulses/sec and 40 000 pulses/sec respectively. Higher velocities are not required in teletraumathology according to the participants. The mean latency of 4 seconds to change arms position with those velocities was judged good enough to use the system with real clinical settings. The graphical interface was judged to be user friendly and precise in showing the reel position of the cameras. All participants were able to reach any desired points of view and could recover full images from field-of-view obstructions in a short amount of time, with minimal lost of concentration on the surgical procedure. Finally, our results show no significant differences between measured variables when surgical procedures were compared from one another. Table 1. Qualitative assessment details at velocities of 90 000 pulses/sec and 40 000 pulses/sec for elbow and shoulder respectively. Number of remote practitioner completely

satisfied Partially satisfied

Not satisfied

quality of image during robot movements

4/6 2/6

quality of image without robot movements

6/6

audible noise

6/6


7

speed to position the cameras

6/6

latency to setup the next position

4/6 1/6 1/6

precision in positioning the cameras

5/6 1/6

user interface satisfaction

4/6 2/6

system’s friendliness

6/6

4. Conclusion In the context of a shortage of trauma surgeons, the development of a tele-coaching system could provide help on demand and consequently, could improve the level of services in trauma medicine. This would be beneficial to the population of regions where trauma surgeons are unavailable, and could decrease medical cost by providing remote support on surgical procedures used to stabilize unstable polytraumatized patients at their arrival in the trauma room. The fundamental goal behind the robot-based tele-coaching system is to reconcile the modes of teleconference and medical examination of polytraumatised patients. It must allow remote surgeons to continuously visualize the general state of the patient while concentrating on the gestures posed by the on-site emergency clinician. We are engaged in an iterative design process in which complete prototypes are designed and rapidly evaluated to outline the appropriate design specifications for novel technology that could provide such services. The first prototype provided a clear proof-of-concept of the potential benefit of such device, with limited range of visual perspectives. The second prototype provided a wider perceptual space and the use of a tactile interface (rather than joysticks), but with a more complex design. The third prototype addresses these limitations and allows us to conduct pre-clinical and clinical testing of the tele-coaching system to assess the performance of the robot design, operator interfaces, transmission of video and audio streams, and the systems usability in trauma interventions.

The next step is to install the system in the trauma room of a regional hospital, the Centre Brome-Missisquoi-Perkins in Québec to perform a quantitative clinical assessment and further usability studies, allowing us to continue improving the system. Then we will conduct additional evaluations with affiliated research centers. Our objectives are 1) to define the modalities for assistance, by a remote trauma surgeon, of an on-site emergency clinician in the practice of trauma medicine and 2) to use tele-coaching as a means to measure the effectiveness of the CHUS tele-training program, as to the effectiveness of having a virtual surgeon assisting the trauma team, compared to on-site trauma surgeon assistance right in the emergency room. Our research initiative constitutes a very promising avenue 1) to develop a standardized real-time tele-coaching procedure, 2) to define legal parameters and image diagnostic quality parameters for transmission through the RTSS (Réseau de télécommunication sociosanitaire du Québec) network which is necessary to preserve security and integrity of data, 3) to improve emergency support on sites deprived of a trauma surgeon, 4) to maintain up-to-date the level of expertise of clinicians, and 5) to develop a surgical procedure program for residency students in surgery and emergency medicine and young physicians. Significant benefits are expected both in terms of patient clinical status whatever the medical facility in which the service is provided: hospitals, health centres, clinics, dispensaries where clinical nurses provide medical support without the help of physicians. Finally, in terms of technology, our long-term objective is to increase the functionality of the technology by: 1) tracking the zone of activity on patients, helping the remote expert surgeon concentrate on the surgical procedure by minimizing camera manipulation [16], 2) develop a new image coding algorithm for improved transmission rate and image quality and 3) proceed to comply with the biomedical and the CSA standards. ACKNOWLEDGEMENTS The authors wish to thank Sono-Vidéo inc., the Faculty of Medicine and Health Sciences and the Faculty of Engineering of the Université de Sherbrooke, and the CHUS for their support in this work. They also wish to thank Denis Bisson


8

and Claudia Beaulieu of the Faculty of Medicine and Health Sciences for their clinical support and physicians who participated in technology assessment. F. Michaud holds the Canada Research Chair in Mobile Robotics and Autonomous Intelligent Systems. References [1] Chan, B.T.B., From Perceived Surplus to Perceived Shortage: What Happened to Canada’s Physician Workforce in the 1990s?, Canadian Institute for Health Information, June 2002, 67 pages. [2] Romanow R.J. Guidé par nos valeurs: L’avenir des soins de santé au Canada, 2002. Ottawa: Santé Canada. [3] Tachakra S., Level of diagnostic confidence, accuracy, and reasons for mistakes in teleradiology for minor injuries, Telemedicine Journal & E-Health, 8(1) :111-121, 2002. [4] Dunn B.E., Choi H., Almagro U.A., Recla D.L., Davis C.W., Telepathology networking in VISN-12 of the veterans health administration., Telemedicine Journal and E-Health, 6(3):349 – 354, 2000. [5] Kennedy C., Yellowlees P., The effectiveness of telepsychiatry measured using the Health of the Nation Outcome Scale and the Mental Health Inventory. Journal of Telemedicine & Telecare, 9(1):12 - 16, 2003. [6] Hicks L.L., Boles K.E., Hudson S.T., Madsen R.W., Kling B., Tracy J., Mitchell J.A., Webb W., Using telemedicine to avoid transfer fo rural emergency department patients, Journal of Rural Health, 17(3): 220-228, 2001. [7] Benger J., A review of minor injuries telemedicine, Journal of Telemedicine & Telecare, 5 Suppl 3 : S5-S13, 1999. [8] Tachakra S., Lynch M., Newson R., Stinson A., Sivakumar A., Hayes J., Bak J., A comparison to telemedicine with face-to-face consultations for trauma management., Journal of Telemedicine & Telecare, 6 Suppl 1: S178-S181, 2000. [9] Tachakra S., Jaye P., Bak J., Hayes J., Sivakumar A., Supervising trauma life support

by telemedicine, Journal of Telemedicine & Telecare, 6 suppl 1 : S7-S11, 2000. [10] Wirthlin D.J., Buradagunta S., Edwards R.A., Brewster D.C., Cambria R.P., Gertler J.P., LaMuraglia G.M., Jordan D.E., Kvedar J.C., Abbott W.M., Telemedicine in vascular surgery: feasibility of digital imaging for remote management of wounds, Journal of Vascular Surgery, 27(6) : 1089-1100, 1998. [11] Rogers F.B., Ricci M., Caputo M., Shackford S., Sartorelli K., Callas P., Dewell J., Daye S., The use of telemedicine for real-time video consultation between trauma center and community hospital in a rural setting improves early trauma care : Preliminary results., Journal of Trauma, vol. 51 : 1037-1041, 2001. [12] Lemieux R., Martin M., Bellemare C., Masson P., Michaud F., Bernard, M.E., Fauteux, P., Julien, C., Lalonde-Filion, M., Morier, C., Morin-Guimond, A., Patry, M.-A., Saint-Pierre, A., Real-time coaching using robot-based semi-teleoperated cameras, Proceedings 3rd Conference IASTED on Telehealth, Canada, 2007. [13] Lemieux R., Masson, P., Bellemarre, C., Martin M., Bernard, M.E., Fauteux, P., Julien, C., Lalonde-Filion, M., Morier, C., Morin-Guimond, A., Patry, M.-A., Saint-Pierre, A., Moisan, P., Michaud, F., Telecoaching in traumatologiy, Proceedings Canadian Medical and Biological Engineering Conference, Canada, 2007. [14] Casper, J. and Murphy, R. Human-robot interactions during the robot-assisted urban search and rescue response at the word trade center. In IEEE Transactions on System, Man, and Cybernetics, Part B, volume 33, pages 367-385, 2003. [15] Olivares, R., Zhou, C. Bodenheimer, B., and Adams, J.A. Interface evaluation for mobile robot teleoperation. In Proceedings ACMSE’03, Savannah, GA, March 2003. [16] Moisan, P., Analyse autonome d’activité de scènes vidéo numériques acquises par un système de télésupervision en traumatologie, Master’s Thesis, Department of Electrical Engineering and Computer Engineering, Université de Sherbrooke, 2006.


9

ROBOTIZED CAMERA SYSTEM FOR REAL-TIME COACHING OF ...

Documents

Transcript of ROBOTIZED CAMERA SYSTEM FOR REAL-TIME COACHING OF ...