Use of Unobtrusive Human-Machine Interface for

Rehabilitation of Stroke victims through Robot

Assisted Mirror therapy

Gautam Narangi, Arjun Narang2, Soumya Singhi Electrical and Electronics Engineering, Bharati

Vidyapeeth's College of Engineering, New Delhi, India Department of Electronics and Instrumentation, Birla

Institute of Technology and Science, Pilani, India gautam2410@gmail.com, arjun.narang09@gmail.com,

soumya.singh 1001@gmail.com

Abstract- Stroke is one of the leading causes of long-term

disability worldwide. Present techniques employed for

rehabilitation of victims suffering from partial paralysis or loss

of function, such as mirror therapy, require substantial amount

of resources, which may not be readily available. In traditional

mirror therapy, patients place a mirror beside the functional

limb, blocking their view of the affected limb, creating the

illusion that both the limbs are working properly, which

enhances recovery by enlisting direct simulation. This paper

proposes an alternate robot based concept, named Wear-A-BAN,

where the rehabilitative task will be carried out by a normal

articulated industrial robot. During the proposed rehabilitative

procedure, the patients are made to wear a smart sleeve on the

functional limb. Movement of this limb is monitored in real-time,

by wireless Body-Area Network (BAN) sensors placed inside the

sleeve, and copied over the sagittal plane to the affected limb.

This procedure results in considerable savings in terms of money

and personnel, as even though this procedure does not make the

rehabilitation process autonomous, but one therapist can monitor

various patients at a time. The industrial robot used is suitable

for this purpose due to safety aspects naturally existing in the

robot, is relatively cheap in price, and allows comprehensive 3-D

motions of the limb. Also, unlike traditional therapy, this

procedure allows actual movement of the affected limb. The

sensors can also be used for other applications, such as gaming

and daily life personal activity monitoring.

Keywords- Stroke; Rehabilitation; Mirror Therapy; Ultra Low

Power WEANs; Sensory nodes; Smart Textile/Fabric

I. INTRODUCTION

With an ageing population in an industrialized world, the global burden of stroke is staggering. Hemiparesis is one of the most common and disabling consequences of stroke [1]. 1 in 6 people worldwide will have a stroke in their lifetime. 15 million people worldwide suffer stroke each year and 5.8 million die from it. Stroke claims a life every 6 seconds. It is the second leading cause of death for people above the age of 60, and the fifth leading cause in people aged 15 to 59 [2]. It is assumed that 40% of all stroke patients benefit remarkably from limb therapy.

978-1-4673-6225-2/13/$31.00 ©2013 IEEE

luhani Lempiainen Managing Director Deltatron Oy Ltd. Helsinki, Finland jle@deltatron.fi

One technique employed to effectively rehabilitate stroke victims, especially those suffering from partial paralysis or loss of function, is using mirror therapy. Mirror therapy is a strategy that has been used successfully to treat phantom pain after amputation and recovery from hemiplegia after a stroke. In traditional mirror therapy, patients place a mirror beside the functional limb, blocking their view of the affected limb, thus creating the illusion that both the limbs are working properly, which enhances recovery by enlisting direct stimulation. In therapy of stroke patients, trials have reported improved prognosis if therapy is started within 20-30 days after the stroke. This time frame often proves to be challenging due to limited personnel resources in healthcare.

In robot assisted mirror therapy, visual illusion and motoring function is enhanced by robot assisted actual movements of the affected limbs. This method has several advantages over conventional methods in terms of clinical and biomechanical measures [3]. During the proposed rehabilitative procedure, the patients are made to wear a sleeve on the functional limb. Movement of this limb is monitored with wireless body-area network (WBAN) sensors placed inside the sleeve. Movement of the functional limb is monitored in real time, and copied over the sagittal plane to the affected limb.

In the Wear-A-BAN project for rehabilitation, the task of moving the affected limb is carried out by a normal 6 degrees of freedom industrial robot for the acute phase brain strokes. The main effort in this case is the rehabilitation of the upper limb and to improve the very basic needs of human independent living. Upper and lower limb robotic tools for traditional neuro-rehabilitation are effective in reducing motor impairment but they are limited in their ability to improve real world function. There is a need to improve functional outcomes after robot assisted mirror therapy. Improvements in the effectiveness of these environments may be achieved by incorporating into their design and control strategies important elements key to induce motor learning and cerebral plasticity

such as mass-practice, feedback, task-engagement, and complex problem solving [4]. Also, from a technological point of view, industrial robots are a cheap, accurate and flexible choice as compared to many alternate approaches.

II. DESCRIPTION OF SENSORY NODES

High intensity, and task specific upper limb treatment consisting of active, highly repetitive movements is one of the most effective approaches to arm- and hand- function restoration. [5] Hence, in mirror therapy with robots, it is essential to know the exact movements of the guiding limb.

For this reason, there are three sensor nodes in the sleeve glove on the guiding limb. There are scientific theoretical models that combine three of these nodes in one limb to be enough to calculate its exact position. The reverse kinematics has been adopted in the project and thus, the limb movements to control the robot can be measured with good accuracy. A low power programmable processor was designed, combining features of a digital signal processor (DSP) and a microcontroller unit (MCU). Implemented as a synthesizable VHDL software intellectual property core, the processor implements a broad range of power saving features including its customizable architecture and reconfigurable instruction set [6]. The Wear-A-BAN project uses the sensory nodes that consist of a low power radio, a system on chip (SoC), a 3D accelerometer, a 3D gyroscope and magnetometer. The main task was to make the information network, Body-Area Network BAN, which can process and transfer a large amount of sensor data wirelessly to computer for further use.

The nodes are programmed separately. The HDK enables easy validation of software for the processor on a SoC circuit. The HDK is composed of a motherboard on top, of which up to 3 daughter boards can be plugged. The board includes a (reprogrammable) circuit, which enables it to connect the daughter connections to the other daughter slots in a different way. Hence, it is possible to add, on the available slots, the application level daughter board. The sensor board is made of all required sensors for the W AB project, and an interface board allows connecting it to the board. The application level drivers have been developed on this system.

Fig.! Central node with receiver antenna

The WBAN sensor nodes used are based on generic design for different scenarios. They incorporate three complementary motion sensors: a 3 DOF digital accelerometer, a 3 DOF digital gyroscope, and a 3 DOF digital magnetometer, the combination of which allows for accurate motion and position sensing. A digital microphone is also used to capture emotional data. The design of the WBAN node was based on the sensor board containing all of the motion sensors, the microprocessor, boot ROM and battery/JT AG adapter and fabric antenna interface, on a PCB of only 20mm x 32mm.

Fig. 2 Complete node system consisting of a generic Wear-A-BAN

module and JTag Programmer board

III. DESCRIPTION OF SMART TEXTILES

"Smart Fabric" is defined as a traditional fabric with integrated with active functionality. "Traditional fabric" includes materials (e.g. cotton, polyester etc.), treatments (e.g. dyes, polyvinyl alcohol, etc.), and manufacturing techniques. Active functionality could include power generation and storage, human interface elements, sensing devices, radio frequency (RF) functionality, or assistive technology [7]. The network will generate research and development programs and will form the framework of new research center in intelligent product research [8]. A key driver for smart textile research is the fact that both textile and electronics fabrication processes are capable of functionalizing large-area surfaces at very high speeds [9].

The technology used to manufacture the textile antenna is weaving technology. The introduction of conductive yams into conventional manufacturing systems (shed weaving and knitting) allows the production of highly-conductive textile structures which are much more permeable than those manufactured using other technologies such as laminating. These fabrics have high color- fastness, flexibility, rigidity, durability and strength. The main drawback of these structures is the difficulties encountered in obtaining complex designs such as the textile antenna which were being developed by this project. This necessitated the search for other complementary technologies in order to ensure the successful outcome of the development. The system has a removable textile antenna. The nodes are placed in sleeve which is worn by the patient on the functional limb. Because all electronic parts and textile antenna are placed inside a pocket, the smart garment will be comfortable, usable, flexible and hence, wearable. The garments are developed using comfortable materials (highly

appropriated for skin contact).The final sensory node with smart textile antenna is shown in Fig.3.

Fig. 3 Complete sensor node with textile antenna

IV. NODE COMMUNICA nON PROTOCOL

The communication protocol enables forwarding of distributed BAN sensor data such that individual data transmitted by each is easily identified, received without error and transmitted to a central device for further processing and visualization. The design of nodes consists of three major goals to enable experimentation: minimal power consumption, easy usage, and increased software and hardware robustness [10]. When relaying functionalities are not used or communications with the central node are good, the W AB has a star topology. Nevertheless, some communications can suffer from shadowing effect and W AB nods can cooperate to provide satisfactory QoS requirements. In this case, the topology becomes a star mesh hybrid topology, where sensor node to sensor node communication is possible.

PC

Driver USB lO COM ----------------------------------- ----------------------------------------

________________________ �:��:��_�::O ���! _________________________ �_;;� __ . UA ramer

Remote Pro rammm A II cation

A lIealion framer

MAC

stracnon

AS ensors WA Radio WAB1Imer

WAS

node

Fig. 4 Communication Protocol Diagram for WBAN

In this communication W AB nodes can have different roles: Coordinator functionalities i.e. the device can send a beacon for synchronization and superframe management, accept or refuse node associations and allocate timeslots for

communications; Central node functionalities for the device connected to the PC; and Sensor node functionalities i.e. W AB nodes have to synchronize on the beacon, to request to he coordinator an association and a BAN specific short address and to request to the coordinator GTS slots.

V. APPLICA nON TO REHABILIT A nON USING WEAR-A-BAN DEMONSTRATOR

Full motion capture for this scenario is illustrated.

• An application scheduling the recuperation of Sensors data is defined.

• The values of W AB sensors is recuperated. • The W AB MAC is used to schedule the

communications. • The W AB radio of the W AB node is used to transmit

the data packet. • The central node receives the data using the W AB

radio and the WAB MAC. • The data packet goes through the different layers

until the UART framer. • The data packet goes from the UART to the USB

thanks to the FTDI of the USB JT AG KEY W AB. • In the PC 1, the data is transferred from the USB to

COM and from the COM to UDP respectively. • In the PC2, the data is used by the W AB GUI using

the UDP port.

� \,. ,I Coordinator

;.

Fig. 5 Complete Node placement

The motion capture demonstrator works as follows: one W AB node is the central node and the coordinator of the network. It is connected to the pc. A second W AB node is a sensor node on the body. This node sends its data in real time according to the movement of the guiding limb during the rehabilitation

scenario. It is assumed that a set-up of 3 sensor nodes is typically enough to calculate the motions of the limb for the control system of the robot by inverse kinematics. This application uses monitoring flow for real time mode, and the expired monitoring data is retransmitted later using a logging mode. The setup during the development process is shown in Fig. 6.

Intelligent

textile sleeve

with 3 node

positions

Fig. 6 Setup during the system development

Graphic User Interface is provided, showing the node on the body, and showing the log when a node is associated, and with a button to start or stop the BAN. The robot used to move the affected limb corresponding to the movement of the guiding limb is ABB IRB140. This is an industrial class robot, and is controlled using ABB Robot Studio. The values of the accelerometer, gyroscope and magnetometer are processed through this software.

ce:J leti

Wear-A-BAN

Fig. 7 Wear-A-BAN demonstrator window

.1 - . �= .... (, ....... 1I-.

Fig. 8 ABB Robot Studio window used for simulation

Fig. 9 Terminal window showing data received at the central node

There are two tasks being done simultaneously by the ABB Robot Studio. The background task sets up a TCP/IP socket communication with the software. This background task will then listen and receive coordinate data (translation and rotation in quaterions) from the software and will analyze this received data. The received data is stored to a robtarget persistent, which will be used by the main task. This main task will check the position of the guiding limb as stored in the robtarget, and move the ABB IRB 140 to that position, if change from previous received position is detected. There are also short wait times at the end of the program to avoid unwanted loop programs.

» 71 1 5 6.0141.010 12.0 -3 .0-5.0-26.0 336.0 -2GlJ.0 -737 .0

• 7 1 : F rame type, moti on se rISe r fr .• me with 3 A3G3 M [Ox47 in h ex adec imal j .

• 1 : Add ress oHh e no de that it h.""e s€ nt t he fram e ·5: Ti me st amp

• ·6 .01 4 .0 1012.0: Ac � e le ro m €t€ rval u e

• -3 .0 -5.0 -26.0: G y rcsm p e val u€

• 336.0 -260.0-737.0: M€g neto met ervalu€

Fig. 10 Real time values of accelerometer, gyroscope and magnetometer received from nodes.

Fig. 1 1 The rehabilitation scenario in real scale

VI. CONCLUSION

This rehabilitation scenario shows, in reality, the good potential of the developed wireless technology in the vicinity of the human body. The limb movements can be monitored and various product concepts can be generated based on this Wear-A-BAN technology. The industrial robot used in Wear-A-Ban is suitable for the proposed rehabilitative purpose due to safety aspects naturally existing in the robot, being relatively cheap in price and allowing comprehensive 3D motions for the arm, all three translations and all three rotations. The robot working envelope, speed and load capacity can easily be adjusted according to the needs of this scenario.

There is no existing concept based on this industrial based technique, but all present rehabilitation machines are complicated, expensive and more or less tailor made for rehab purposes. The intelligence capability of the industrial robot can be utilized while adapting the machinery for the different body sizes of the patients, and also for compensating the physiological non-Iinearities of the limb movements in a human body. This is a case in, for example, the human shoulder, where the upper limb doesn't rotate around one fixed point, but withdraws horizontally according to the height of the limb. When calculating the reverse kinematics for the robot according to the sensory data, these non-linearities can be taken into account without any physical mechanisms. The measurement of the rehabilitated limb's movements can be easily displayed and the progress of the rehabilitation can thus be monitored on PC software. This concept cannot make the therapy autonomous, but one therapist can work with several patients at the same time and this increases rehabilitation capacity and power. Beside visual illusion, it also provides a tactile sense of movement of the affected limb.

The potential end-users of this robot based rehabilitation scenario will be interested to further develop and rest-run the robotized rehabilitation concept. The developed technology has now been proven to work in this environment.

The main hurdles to overcome in the developed sensor prototype are:

• Battery capacity must be increased. • Accelerometer sensor sensitivity must be fine-tuned

for this application • Gyroscope sensor must be tuned to achieve less

power consumption.

VII. FUTURE APPLICA nONS AND DEVELOPMENT

The rehabilitation system powered by industrial robot is a promising concept. The growth in this market segment is obvious because of the new intelligent technology, and

increase in the number of people who need rehabilitation. Commercialization of the robot based therapy concept to many physiotherapy modalities gives room for more markets. The use of robot as an aid tool in many other kind physiotherapy solutions will be profitable. When the price of sensor node technologies goes down along with a long production series, the robot assisted rehabilitation technology will spread into physiotherapy clinics. The new body area network communication technology opens new solutions for new modalities for Human-Machine interaction (HMI). For example, hand movements could be used as a gesture language.

ACKNOWLEDGMENT

The Wear-A-Ban project was funded by the EU's FP7 Research for the benefit of SMEs Associations, under contract number 243473. The developed sensor techniques are available at Deltatron Ltd. in Helsinki, Finland.

REFERENCES

[1] Altschuler, E.; Wisdom, S.; Stone, L.; Foster, c.; Galasko, D.; Llewellyn, M.; Ramachandran, V.; "Rehabilitation of hemiparesis after stroke with a mirror", The Lancet, 2035-2036, 1999 [2] http://www. world-stroke.org/ [3] Lum, P. S.; Burgar, c.; Shor, P.; Majmundar, M.; Van der Loos, M.; "Robot Assisted Movement Training Compared with Conventional Therapy Techniques for Rehabilitation of Upper-Limb Motor Function after Stroke", Archives of Physical Medicine and Rehabilitation, 952-959, 2002 [4] Johnson, M.J.; "Recent trends in robot-assisted therapy environments to improve real life functional performance after stroke", Journal of NeuroEngineering and Rehabilitation, 2006 [5] Prange, G.; Jannink, M.; Groothuis-Oudshoorn, c.;

Hermens, H.; IJzerman, M.; "Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke", Journal of Rehabilitation Research and Development, 171-184, 2006 [6]Arm,C., Gyger, S. ; Masgonty, J.-M.; ; Nagel, J.-L. ; Piguet, c.; Rampogna, F. ; Volet, P. "Low-Power 32-bit Dual-MAC 120 W/MHz l.0 V icyflexl DSP/MCU Core", IEEE Journal of Solid State Circuits, 2055-2064, 2009 [7] Simon, c.; Potter,E.; McCabe, M. ;Beggerman, c.; "Smart Fabrics Technology Development", NASA innovation fund project, 2010

[8] Baurley, S.L. "Smart Textiles for future intelligent consumer products", lEE Eurowearable, 73-75, 2003 [9] Cherenack, K.; van Pieterson, L.; Smart Textiles: Challenges and Opportunities", Journal of Applied Physics, Volwne 112, Issue 9, 2012

[10] Polastre, J.; Szewczyk, R.; Culler, D.; Telos; "Enabling ultra-low power wireless research", 4th International Symposium on Information Processing in Sensor networks, 364-369, 2005