Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single...

69
Robo-Wrist Controller by Mohamed Athiq Faraquddin Ahamed B. Tech, Anna University, 2010 Project Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Engineering in the School of Engineering Science Faculty of Applied Sciences ©Mohamed Athiq Faraquddin Ahamed, 2015 SIMON FRASER UNIVERSITY Summer 2015 All rights reserved. However, in accordance with the Copyright Act of Canada, this work may be reproduced, without authorization, under the conditions for Fair Dealing. Therefore, limited reproduction of this work for the purposes of private study, research, criticism, review and news reporting is likely to be in accordance with the law, particularly if cited appropriately.

Transcript of Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single...

Page 1: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

Robo-Wrist Controller

by

Mohamed Athiq Faraquddin Ahamed

B. Tech, Anna University, 2010

Project Submitted in Partial Fulfillment of the

Requirements for the Degree of

Master of Engineering

in the

School of Engineering Science

Faculty of Applied Sciences

©Mohamed Athiq Faraquddin Ahamed, 2015

SIMON FRASER UNIVERSITY

Summer 2015

All rights reserved. However, in accordance with the Copyright Act of Canada, this work may be reproduced, without authorization, under the conditions for Fair Dealing. Therefore, limited reproduction of this work for the purposes of private study, research, criticism, review and news reporting is likely to be in

accordance with the law, particularly if cited appropriately.

Page 2: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

ii

Approval Name: Mohamed Athiq Faraquddin Ahamed Degree: Master of Engineering Title of Project: Robo-Wrist Controller Examining Committee:

Chair: ______________________________________

Dr. Ljiljana Trajkovic

Professor, Engineering Science, SFU

______________________________________

Dr. Carlo Menon Senior Supervisor Assistant Professor, Engineering Science, SFU

______________________________________

Dr. Fabio Campi Supervisor Lecturer, Engineering Science, SFU

______________________________________

Dr. Craig Scratchley Internal Examiner Senior Lecturer, Engineering Science, SFU

Date Defended/Approved: April 23rd, 2015________________

Page 3: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

iii

Abstract

Robo-wrist controller is a Graphical User Interface (GUI) based application designed and

developed to control, and monitor a wrist exoskeleton driven by micro-controllers. The

software application is developed to better assist a physiotherapist in administering physical

therapy to stroke patients with the help of the exoskeleton. Since the micro-controller device

that directly controls the exoskeleton is not very intuitive to use and lacks visual feedback, a

software application fills these gaps and makes the device more user-friendly. It also provides

the benefits of a wireless controller. The software application comprises two major aspects: an

Android application for use in smartphones and a Windows PC version. The major differences

between these two are the number of devices that can be controlled simultaneously and the

communication media. The android application can control at most one device at any given

time via Bluetooth, while the PC version is capable of controlling multiple devices

simultaneously because it uses both Bluetooth and Wi-Fi communication media. Both types of

software applications have features such as: ability to add new devices, view the added devices,

create an exercise protocol and store it, view stored protocols, a data visualizer to plot

incoming feedback data from the exoskeleton, and multiple user profiles with different levels of

administrative privileges. Each module in the applications is independent of each other as well

allowing for easy modifications and additions to the application in the future. The GUI of the

software makes it very easy for the therapist to use the exoskeleton using visual controls and

feedback allowing for easy administering of physical therapy using the exoskeleton to his/ her

patients.

Keywords: Exoskeleton, wireless control, GUI application

Page 4: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

iv

Acknowledgements

I would like to thank Dr. Carlo Menon for his guidance and support throughout

this project work. He has given me a tremendous opportunity to discover my

talents and apply my knowledge in a real-world application. I also would like to

thank Dr. Fabio Campi for his support and guidance on this project.

I would like to thank my colleagues Mr. Arabi Elhouderi for designing and building

the micro-controller device without which my application would not have borne

fruition. I would like to thank Mr. Yiming for building the exoskeleton to test the

micro-controller and the software applications.

I would like to thank physiotherapist Ms. Bubblepreet Randhawa for all her

involvement, support, suggestions, and feedback in building this software.

I would like to thank all my colleagues at MENRVA lab for giving me moral and

technical support.

I would also like to thank Bill Gates, Tim Berners Lee, Larry Page, and Sergey Brin

for their contributions to the world of technology.

Page 5: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

v

Table of Contents

Approval ii

Abstract iii

Acknowledgements iv

Table of Contents v

Chapter 1: Introduction 1

1.1 Motivation 1

1.2 Objectives 2

1.3 Sections 2

Chapter 2: Literature Review 3

Chapter 3: Project Overview 5

3.1 The Exoskeleton 5

3.2 The Microcontroller Box 6

3.3 The Software application 7

3.3.1 Android application 8

3.3.2 Desktop PC application 11

3.4 How Robo-wrist system works? 17

Chapter 4: System Design 19

4.1 The Exoskeleton 19

4.1.1 Parts of the Robo-wrist 21

4.1.2 Functions of the Robo-wrist 21

4.2 The Microcontroller box 23

4.2.1 Communications module 24

4.2.2 Functions of the communications module 25

4.2.3 Microcontroller board 25

4.2.4 Functions of the microcontroller 26

4.2.5 Motor driver Unit 27

4.2.6 Protective case 29

4.2.7 How the microcontroller box works? 29

4.3 The Software application 31

4.3.1 Android application 31

4.3.1.1 Design model 32

4.3.1.2 Background processes 32

Page 6: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

vi

4.3.1.3 Pages/Modules 32

4.3.1.4 Login page 33

4.3.1.5 Home page 34

4.3.1.6 Data Visualizer 35

4.3.1.7 Known Devices page 37

4.3.1.8 Available Protocols page 38

4.3.1.9 Create protocol page 38

4.3.1.10 Start Exercise page 41

4.3.1.11 Test plot page 41

4.3.2 Desktop PC application 44

4.3.2.1 Design model 44

4.3.2.2 Background processes 44

4.3.2.3 Modules 46

4.3.2.4 Login module 47

4.3.2.5 Home page 49

4.3.2.6 Data Visualizer 49

4.3.2.7 Available Devices module 50

4.3.2.8 Available Protocols module 52

4.3.2.9 Create Protocol module 53

4.3.2.10 Bluetooth Exercise module 54

4.3.2.11 WiFi Exercise module 54

4.3.2.12 Multi Visualizer 57

4.4 Arduino Control Algorithm 58

Chapter 5: Conclusion 60

References 61

Page 7: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

1

CHAPTER 1 INTRODUCTION

1.1 Motivation

Statistics Canada has ranked stroke to be the third largest cause of death in Canada in its

reports from 2000, 2010, and 2011 [1]. Of the total, 40,000 to 50,000 strokes in Canada

each year [2], 13,000 to 15,000 of Canadians die [1], while the rest suffer from minor

recoverable impairment to non-recoverable permanent impairment requiring long-term

care according to Heart and Stroke Foundation of Canada [3]. The Public Health Agency

of Canada (PHAC) estimates about 315,000 Canadians suffer from effects of stroke in

their 2009 and 2011 reports [4, 5].

With stroke affecting so many Canadians annually, rehabilitation is all the more important

as part of providing good health care to the patients and early care for Stroke patients has

a significant impact on the patients’ functional independence than delayed care [8]. Mahler

et al., [7] in their study have shown that 81% of Stroke patients have recovered and live

independently after 6-12 months of Stroke rehabilitation. In a typical Stroke rehabilitation

program Gait training, Balance training, and Strength training play an essential part in

restoring physical functions of the patients [6]. Occupational therapists and

physiotherapists are the major care providers in these training programs.

While physiotherapists are sought after in the Canadian healthcare industry, a 2006

census report from Statistics Canada shows a total of 18,095 registered physiotherapists

Canada-wide [9]. A 2010 report by Canadian Institute for Health Information limits that

number between 16,419 to 16,885 from the years 2007 to 2010 [10]. The statistics show

that ratio of physiotherapists to patients is very low and is decreasing rapidly with the

ageing population on the increase in addition especially in BC.

In order to bridge the gap between increasing number of patients and inadequate number

of physiotherapists a new solution involving robotic exoskeletons is proposed. In this

proposal, a mechanical robotic exoskeleton (Robo-wrist) designed to fit the human wrist

and move to provide physiotherapy is used. To control the movement of the mechanical

Robo-wrist, a microcontroller based control box is attached via cables to the wrist. The

control and monitoring of the Robo-wrist is accomplished via a software application which

Page 8: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

2

is GUI based for ease of use and rea-time visual feedback. The setup will be used with

active supervision by the physiotherapist to administer physiotherapy. The advantages of

this Robo-wrist lie in the fact that it is wirelessly controlled by the software and one can

control and monitor multiple Robo-wrists from the software application. The software

application can be used either in a smartphone platform or a desktop computer platform.

1.2 Objectives

There are two major components of this project and each has its own objectives. The

major components are an Android based smartphone application and a desktop software

application.

Android application:

1. Access control

2. Create and browse exercise protocols

3. Add new devices

4. Start and stop the exercise anytime

5. Communication via Bluetooth

6. Single device Data visualizer

7. Single Robo-wrist control/ monitor at any given time via Bluetooth.

Desktop application:

1. Access control

2. Create and browse exercise protocols

3. Add new devices

4. Start and stop the exercise anytime

5. Communication via Bluetooth and WiFi

6. Multiple device Data visualizer

7. Single Robo-wrist control/ monitor at any given time via Bluetooth

8. Multiple Robo-wrist control/ monitor via WiFi.

1.3 Sections

This report is divided into the following sections: Introduction, Literature review, Project

Overview, System Design, and Conclusion.

Page 9: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

3

CHAPTER 2 LITERATURE REVIEW

Powered Exoskeletons have been in research and testing since the 1960’s with GE

building the first usable one named Hardiman [11] designed to augment human strength.

Since then, various agencies across the world have produced their versions, but notably

for military use. Some of them worth mentioning are Lockheed Martins’ HULC [12, 22],

Cyberdyne’s HAL [13], Raytheon’s XOS [14], EksoBionics’s Ekso [15], ReWalk Robotics’s

ReWalk [16], University of California – Berkeley’s BLEEX [17], ExoHiker [18], ExoClimber

[19], eLegs [20], and Austin [21]. Titan Arm [23], X-Ar [24], and WREX [25] from JAECO

are also worth mentioning.

HULC or Human Universal Load Carrier is a completely untethered hydraulic powered

system capable of carrying up to 200 pounds of payload for extended periods while XOS

is built to be agile enough to let the wearer play soccer in addition to augmenting user’s

strength. BLEEX (Berkeley Lower Extremity Exoskeleton), ExoHiker, and ExoClimber are

all incremental versions of Lower extremity exoskeletons designed to augment user’s

strength during strenuous activities and reduce fatigue. eLegs and Austin on the other

hand are designed to help people with mobility disorder due to Stroke, Spinal injuries or

muscle related disorders. Ekso, ReWalk, and HAL are also designed for clinical use in

physiotherapy. These clinical versions are used as physical assist devices in addition to

rehabilitation therapy. These focus solely on mobility and are mostly developed for military

use.

Powered exoskeletons have come a long way from being potentially hazardous to the user

to being used in daily activities without supervision. The examples mentioned above are

all very well designed and very user friendly but they do have inherent drawbacks as well.

They are all self-contained systems with batteries motors, hydraulics and such. One

common feature among all these is they are bulky due to big batteries, require backpack

like system to house the computers and are expensive. These afore mentioned robots are

all designed for mobility while Titan Arm, X-Ar, and WREX are designed for upper limb

strength augmentation. Most of these upper limb systems focus solely on lifting things and

not on finer movements like the wrist or fingers which are one of the major areas affected

in Stroke patients.

Page 10: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

4

To address this issue of providing rehabilitation for finer functions using the wrist and the

fingers Robo-wrist was conceived. This system does not require bulky backpacks or

computers to operate and the cost is also kept to a minimum to be able to be used by

patients for personal use. In addition, the software interface provides valuable information

that can be used for monitoring and tracking progress in therapy. Being able to control

multiple Robo-wrist simultaneously also gives the physiotherapist the ability to care for

multiple patients at the same time.

Page 11: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

5

CHAPTER 3 PROJECT OVERVIEW

The complete Robo-wrist system in its essence can be divided into three main

components with the software application being the mainstay of this project. The three

major components are:

1. Robo-wrist (exoskeleton)

2. Microcontroller box

3. Software application

The working of the system is shown in Figure 1 and the modules are described in the

sections following. The system works as follows: The exoskeleton is a mechanical device

which is attached to the subject’s forearm that requires physiotherapy securely. The micro

controller box is attached via detachable cables to the exoskeleton and it sends power

and signals to operate the motors on the exoskeleton. The software application provides

a graphical user interface to the user to specify the parameters for a particular type of

exercise for the wrist. The software application sends these parameters as commands

wirelessly to the microcontroller.

3.1 The Exoskeleton

The exoskeleton is built by 3D printing a Solidworks [27] 3D CAD design model using ABS

plastic in a 3D printer. The 3d printed parts are then assembled manually. The exoskeleton

is driven by means of motors built into it and is powered and controlled by the

microcontroller box. There are also potentiometers in the exoskeleton to monitor the

movements of the exoskeleton and send the data as feedback to the microcontroller. The

exoskeleton is attached to the microcontroller box via cables and the cables can be

detached upon need. A detailed view is shown in the Figure 2.

Page 12: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

6

Figure 1. The complete system working principle.

3.2 The Microcontroller box

The microcontroller box is used to control the exoskeleton using electrical signals and is

attached to the exoskeleton via detachable cables. The microcontroller box comprises of

an Arduino Mega 2560 microcontroller board attached to a custom circuit board housing

the motor drivers, LCD screen, buttons and the Li ion battery. These circuit boards are

enclosed in a 3D printed ABS plastic case. The Arduino board is also connected to a HC-

06 Bluetooth module and an ESP 8266 WiFi module for wireless communication with the

software application.

Robo-wrist

Micro-controller

box

Windows PC application

Android application

Page 13: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

7

Figure 2. Robo-wrist exoskeleton. Top row: top view. Middle row: side view. Bottom row:

front view.

3.3 The Software Application

The software application sends and receives data to the microcontroller box wirelessly. It

also provides a Graphical User Interface (GUI) for ease of use. There are two major

aspects of the software application: Desktop PC application and Android smartphone

application.

Page 14: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

8

Figure 3. Microcontroller box showing LCD screen, buttons, 3D printed case and

detachable cable to connect to the exoskeleton.

3.3.1 Android application

The Android application is designed to run on any smartphone running Android version

4.0 or higher. The Android application gives more versatility and portability to the software

as well providing convenience of a handheld device. To begin with the Android application

communicates through Bluetooth to the microcontroller box and hence is able to connect

to utmost one microcontroller box at any given time to send and receive data. The android

application follows a page design and each module is considered a page. The pages/

modules in the android application are:

1. Login/ welcome

2. Homepage/ Main page

3. Data visualizer

4. Known devices

5. Create new Exercise protocol

6. Available Exercise protocol

7. Test plot

8. New Exercise

Login page – Provide access to the application via pre-authorized credentials.

Page 15: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

9

Homepage - Provides access to various other modules to control and monitor the

exoskeleton via the microcontroller.

Data Visualizer – Visualize feedback data from the microcontroller using a real-time graph

plot.

Known devices – View a list of Bluetooth paired devices or previously known devices for

quick verification.

Create protocol – Create and store a new exercise protocol in the phone’s internal

memory. An exercise protocol is a file containing data on how the exoskeleton should

move.

Available protocol– Browse for previously created exercise protocols.

Test Plot – Test graph plot window to check if the graphing module is running properly.

New Exercise – Start an exercise by connecting to one of the known devices, sending a

exercise protocol file and pressing start button. Also has stop, emergency stop

capabilities.

All the above modules are shown in figures 4 to 7.

Figure 4. Modules. Login/ Welcome screen (left), Homepage (right).

Page 16: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

10

Figure 5. Modules. Known devices (left), Available exercise protocols (right).

Figure 6. Modules. Data Visualizer (left), Create exercise protocol (right).

Page 17: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

11

Figure 7. Modules. Test plot (left), New exercise (right).

3.3.2 Desktop PC Application

The desktop pc application is designed to run on computers running Windows 7 or higher.

The desktop application gives more freedom to the user as desktops are more commonly

found. Another advantage is the bigger screen size due to which multiple visualizations/

graphs can be plotted in one screen. The desktop application has all the features as the

Android application and a WiFi mode to connect to multiple exoskeletons via multiple

microcontrollers simultaneously. The desktop application follows a panel design where the

software window has two panels: a static panel with buttons on the left and a dynamic

panel on the right. The right panel changes corresponding to the button pressed on the

left panel. The modules in the desktop application are:

1. Login/ welcome

2. Homepage/ Main page

3. Data visualizer

4. Known devices

5. Create new protocol

Page 18: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

12

6. Available protocol

7. Bluetooth Exercise

8. WiFi exercise

9. Multi visualizer

Login page – Provide access to the application via pre-authorized credentials.

Homepage - Provides access to various other panels/ modules to control and monitor the

exoskeleton via the microcontroller.

Data Visualizer – Visualize feedback data from a single microcontroller using a real-time

graph plot.

Available devices – View a list of previously known Bluetooth and WiFi devices for quick

verification.

Create protocol – Create and store a new exercise protocol in the computer’s hard drive.

An exercise protocol is a file containing data on how the exoskeleton should move.

Available protocol – Browse for previously created exercise protocols.

Bluetooth Exercise – Start an exercise by connecting to one of the known Bluetooth

devices, sending an exercise protocol file and pressing start button. Also has stop,

emergency stop capabilities.

WiFi Exercise - Start an exercise by connecting to one of the known WiFi devices,

sending an exercise protocol file and pressing start button. Also has stop, emergency stop

capabilities. In this window up to two devices can be connected and visualized.

Multi visualizer – Window with the capability to connect to four WiFi devices

simultaneously and visualize in real-time.

All the above modules are shown in figures 8 to 16.

Page 19: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

13

Figure 8. Login page for desktop application.

Figure 9. Home page for desktop application. Left panel is the static panel with buttons

while the right blank space is the dynamic panel.

Page 20: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

14

Figure 10. Available Devices panel showing the previously known or connected

Bluetooth devices and WiFi devices.

Figure 11. Available Exercise protocols panel showing the previously created protocols

in a list and the selected protocols content on the right.

Page 21: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

15

Figure 12. The panel that allows the user to create a new protocol and save it.

Figure 13. Data Visualizer panel with a graph plot.

Page 22: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

16

Figure 14. Bluetooth exercise panel with a visualizing graph plot.

Figure 15. WiFi exercise panel showing two sub panels to connect and visualize two

WiFi devices simultaneously.

Page 23: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

17

Figure 16. Multi visualizer panel with four sub panels to visualize four WiFi devices

simultaneously.

3.4 How Robo-wrist system works?

The Robo-wrist system is a complete self-contained system comprising the exoskeleton

or the Robo-wrist itself, the microcontroller box, and the software application. The way this

system works is as follows.

1. First, the Robo-wrist is securely fitted to the subject’s wrist and forearm using the

forearm braces, palm strap and the Saebo device [26].

2. The microcontroller box is attached to the Robo-wrist via the detachable cables

and powered on.

3. The software application either on the smartphone or the desktop is then run.

4. In the Android application Bluetooth devices are searched, the required one is

selected and connected. A specific exercise protocol file is selected from the

available list and sent to the microcontroller. This exercise protocol file has the

parameters specifying the range of motion for the Robo-wrist including; Extension/

flexion, radial deviation, ulnar deviation, speed of motion, number of repetitions in

each direction. If needed a new exercise protocol can be created on the fly and

Page 24: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

18

selected. Then Start/ stop buttons are pressed for starting or stopping the exercise

respectively. The feedback can be visualized in the data visualizer by connecting

to the device of interest.

5. In the Desktop application, both Bluetooth and WiFi devices are searched upon

application start. The required device is then selected and connected. A exercise

protocol file is either created or selected from the available list of files and sent to

the micro controller. A set of start/ stop buttons aid in starting or stopping the

exercise. The feedback can be visualized in the data visualizer by connecting to

the device of interest.

6. After the completion of physiotherapy, the microcontroller box can be turned off,

detached from the Robo-wrist, and stored. The software application can be closed

or logged off for future use.

Page 25: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

19

CHAPTER 4 SYSTEM DESIGN

In this chapter, an in depth description of the system and its functions are presented. To

begin with we will discuss the components of the Robo-wrist system in detail.

4.1 The Exoskeleton

The Robo-wrist is an exoskeleton designed to replicate the functions of a healthy

functioning human wrist. This is very essential to the whole physiotherapy process since

the objective of using this Robo-wrist is to restore normal functions of an affected wrist.

The human wrist is capable of doing very complex movements with three degrees of

freedom. The Robo-wrist is designed to replicate the human wrist as much as possible,

so it has three degrees of freedom as well but the motion is restricted to two planes to this

point to keep the physiotherapy simple. In a human wrist there are six types of primary

movements involved: Extension, flexion, pronation, supination, radial deviation, and ulnar

deviation. These movements are visually represented in Figure 17. The Robo-wrist’s

movements replicating Extension and flexion are shown in Figure 18 while the radial and

ulnar deviation movements are shown in Figure 19. To operate the Robo-wrist, the

exoskeleton is fixed to the subject’s hand firmly using the forearm braces and adjusting

the palm strap for a comfortable and secure fit. The fingers are then inserted into the

SaeboFlex [26] device.

Page 26: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

20

Figure 17. Movements of the human wrist.

Figure 18. Extension/ Flexion of the Robo-wrist.

Figure 19. Radial and Ulnar deviation of the Robo-wrist.

Extension

Flexion

Fixed part of the

Robo-wrist

Ulnar Deviation

Radial Deviation

Fixed part of the

Robo-wrist

Page 27: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

21

4.1.1 Parts of the Robo-wrist

Broadly the Robo-wrist comprises of two parts: the fixed forearm brace and the moving

wrist assembly.

The fixed part comprises,

1. Forearm brace

2. Cables to connect to the microcontroller

3. Emergency stop button

4. Motor, gear, and potentiometer assembly for radial and ulnar deviation motion.

The moving assembly comprises,

1. Palm strap

2. Up/ down moving frame

3. SaeboFlex device for the fingers

4. Motor, gear, and potentiometer assembly for extension and flexion motion

5. Left/ right moving frame.

The SaeboFlex device [26] is an external passive device that is commercially available to

exercise the fingers. It is disassembled, modified, and bolted directly onto the Up down

moving frame to provide exercise for the fingers as well. This modification complements

the Robo-wrist by eliminating the need to use additional devices to exercise the fingers.

4.1.2 Functions of the Robo-wrist

The Robo-wrist currently performs two sets of motion in two planes. The first one is

Extension and flexion in the y-plane and the second one is Radial deviation and Ulnar

deviation in the x-plane. The motion is initiated and performed by motors built inside the

Robo-wrist. The amount of movement and position of the moving assembly in either

direction is measured, monitored and sent as feedback to the microcontroller using the

built-in potentiometers.

Page 28: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

22

Figure 20. Robo-wrist side view with its constituent parts.

Fixed

Forearm

brace

Saebo

device

Palm

Strap

Left/ right motor

and Potentiometer

Up/ down motor

and Potentiometer

Emergency

Stop

button

Up/

Down

moving

frame

Left/ right

moving frame

Page 29: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

23

Figure 21. Robo-wrist top view with its constituent parts.

4.2 The Microcontroller box

The microcontroller box is an essential part of the Robo-wrist system as it is the control

centre that sends power and control signals to the motors in the exoskeleton. As simple

as it seems there are some constituent parts of the microcontroller box which are

responsible for different function. They are

1. Communication module

2. Microcontroller board

Saebo

device

Up/

down

motor

and

Potenti-

ometer

Forearm

brace

Emergency

Stop

button

Cable to

Microcontroller

Left/ right

motor and

Potentiometer

Left/ Right

Moving

Assembly Up/

Down

moving

frame

Page 30: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

24

3. Motor driver board

4. Power supply/ battery.

Figure 22. Microcontroller box components.

4.2.1 Communications module

The communications module used in the microcontroller box are of two types: Bluetooth

module and a WiFi module. The Bluetooth module used is an HC-06 plug and play module

and the WiFi module used is an ESP8266 plug and play module. These modules were

chosen for their reliability and ease of use among other modules like the Bluetooth and

WiFi shields. These modules communicate directly with the software application. The

Bluetooth module follows Bluetooth 3.0 version and the WiFi module uses IEEE 802.11.

Battery

Motor

driver

unit

Microcontroller board

Communication

module

Exoskeleton Software

application

Page 31: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

25

Figure 23. Communication modules. WiFi ESP 8266 (left), Bluetooth HC-06 (right).

4.2.2 Functions of the Communication module

1. Listen to incoming connections (acts as a server)

2. Establish connection upon request from the software application

3. Send data from the microcontroller to the software application

4. Receive commands and data from the software application

4.2.3 Microcontroller board

The microcontroller board is the brains behind the microcontroller box. In this system an

Arduino Mega 2560 microcontroller board is used. It has 54 pin outs providing digital,

analog, and PWM outputs, 256kb of memory and 16 Mhz clock speed. It is also USB

capable for programming and also for power.

Page 32: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

26

Figure 24. Arduino Mega 2560 microcontroller board.

4.2.4 Functions of the Microcontroller

1. Provide power to the communication module and the motor driver board

2. Send digital, PWM signals to the communication module and the motor driver

board

3. Continuously read values from digital pins connected to the potentiometers and

motors

4. Process incoming commands and data from the software application and do the

corresponding task like start/ stop the motors, assign values to minimum and

maximum range of the motion for the exoskeleton, and convert degree values to

potentiometer values.

5. Prepare data to send to the software application

6. Create variables to store values, monitor flags and timers, and run continuous

running loops.

USB port

Pinouts

Reset button 2.1 mm DC power

Page 33: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

27

4.2.5 Motor Driver Unit

The motor driver unit is a custom fabricated circuit board which acts as the nervous system

of the microcontroller box delivering power, PWM signals, sending and receiving data to

and from the motors and potentiometers. This circuit board is built to be sandwiched with

the Arduino Mega 2560 microcontroller board and it houses motor driver ICs, pins for LCD

screen, communication modules, power supply, buttons, and pins for connecting the

Robo-wrist.

Figure 25. Motor driver board component parts.

Pinout to

Robo-

wrist

Motor

driver ICs

Joystick

On/ Off

switch

Li ion

battery

Page 34: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

28

Figure 26. Motor driver circuit board schematic diagram.

On/ Off

switch

Motor driver

ICs

Pinout for

Robo-wrist

Page 35: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

29

4.2.6 Protective Case

The microcontroller box components are all assembled and are enclosed in a protective

case which is designed using Solidworks and printed using ABS plastic in a 3D printer.

The case is small enough to be held in one hand and used similar to a phone. It has

contoured edges, ergonomic curves on the back for comfort. The case is designed in two

pieces and can be snap shut without the help of any screws or other fixtures and is also

easy enough to be opened for maintenance. The dimensions of the case are 120x72x25

mm.

Figure 27. 3D printed protective case for the microcontroller components.

4.2.7 How the microcontroller box works?

The operational flow chart of the microcontroller box is shown in Figure 28.

Page 36: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

30

Figure 28. Microcontroller box operation flowchart.

Power On

Motor drivers Power

On motors

Microcontroller Initialize

variables, flags

Comm. Module listens

for connections

Connection

established

Idle Idle

Data received Process data; assign minimum and

maximum range, total number of

repetitions, speed of motors

Idle

Start command

received

Send PWM to motor to

move to minimum position

Idle

Move to minimum position

If minimum position reached

Send PWM to motor to

move to maximum position

Total reps

achieved?

Move to maximum

position

If maximum position reached

Send PWM to motor to move

to minimum position

Move to minimum

position

Increment Reps done by 1.

Send

potentiometer

data to

software

application

Yes

No

Stop PWM signal Stop motors Stop command

received

Process command

Page 37: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

31

4.3 The Software Application

The software application is the most visually interactive part of the Robo-wrist system and

the two components of the software application are Android application and Desktop PC

application. The android application is designed to work on smartphones running Android

version 4.0 or higher which covers the majority of Android market share. The Android

application was designed with the LG Nexus 5 phone for testing, so it is best viewed on a

5 inch phone screen. The desktop PC application is designed to work on desktop

computers running Windows version7 or higher. Windows operating system occupies

majority market share in the personal computer system. The applications are feature rich

and are very intuitive to use. The two core aspects of the software application design are

modularity and extensibility these are strongly reflected in both the Android and desktop

applications.

Extensibility

In software design, extensibility refers to thinking ahead in terms of the software

application’s future. This is part of the evolutionary model of design and development

which emphasizes on incorporating mechanisms for the software to evolve based on end-

user’s changing needs and accommodates changes.

Modularity

Modularity refers to compartmentalizing functions into independent and interchangeable

modules. This leads to easier modifications, additions, and deletions of the software

application according to the needs of the end user.

4.3.1 Android application

The first part of the software application is the Android smartphone application designed

to connect, control and monitor the Robo-wrist using Bluetooth communication protocol.

This allows the Android application to connect and control one Robo-wrist at a given time.

The Android application is limited to one device connectivity, since connecting multiple

devices requires multiple visualizations and a small screen is not ideal for such task. The

Android application was developed first as a test bed to test the feasibility and usability of

such an application.

Page 38: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

32

4.3.1.1 Design model

The Android application follows the design principle of page based application where each

module is considered a page or a separate activity. The figure 29 illustrates the data flow

in the page based design.

Figure 29. Android application’s page design where two button clicks are required to

switch between pages.

4.3.1.2 Background processes

The application when started is designed to perform some background processes

essential for the smooth functioning of the application. The processes performed are

1. Turn on Bluetooth module in the phone

2. Search for known or previously paired Bluetooth devices from the phone’s

Bluetooth device register.

3. Access the application’s folder in the phone’s internal memory or SD card and

read all the exercise protocol files created during the previous application use.

4.3.1.3 Pages/ Modules

The Android application’s structure is comprised of functions bundled up in individual

modules called pages or activities. The pages incorporated are given below.

1. Login page

Login page/

module

Home Page/

main module

Data

Visualizer

page

Known

Devices

page

Available

Protocols

page

Create

Protocol

page

Start

Exercise

page

Test

Plot

page

Forward flow

Backward flow

Page 39: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

33

2. Homepage

3. Data visualizer page

4. Known devices page

5. Available protocols page

6. Create protocols page

7. Start Exercise page

8. Test plot page

4.3.1.4 Login page

The login page serves as the first screen upon application start and this module is

responsible for access control. Based on different login details multiple levels of access to

the application can be implemented. In the current scenario, there are two user profiles/

access; guest and administrator access. The guest access limits the user to access only

the visualizer page while the administrator access allows access to all the pages.

Figure 30. Login page of the Android application.

Page 40: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

34

Figure 31. Android application access control flow chart.

4.3.1.5 Home page

The home page of the application is the central hub for traversing through various pages

or modules. To facilitate intuitive use, this page is designed as a button based palette

where each button when pressed takes the user to the respective page indicated but the

name on the button. This design was opted for its simplicity, extensibility and intuitiveness.

Login page

Home Page

Data

Visualizer

page

Known

Devices

page

Available

Protocols

page

Create

Protocol

page

Start

Exercise

page

Test

Plot

page

Data

visualizer

page

Message

prompt

Guest login

Admin login

Invalid login Check

login

Page 41: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

35

Figure 32. Android application homepage with buttons for respective pages.

4.3.1.6 Data Visualizer

The data visualizer is the module responsible for plotting feedback data from the Robo-

wrist potentiometers via the microcontroller box in real-time. In this module the user can

select device he wants to connect through a drop down list, connect, and the graph module

will automatically start plotting the feedback values if the Robo-wrist device is sending

feedback data.

Page 42: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

36

Figure 33. Data Visualizer page for the Android application displaying real-time plotting

of potentiometer data from the Robo-wrist moving from left to right to left.

Page 43: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

37

4.3.1.7 Known devices page

The known devices page shows the user a list of known Bluetooth devices previously

used. This list is also shown in other pages when the user tries to connect and perform

exercises or visualizations. If a new device is to be connected, the user manually goes to

the Phone’s Settings menu and adds/ pairs the Bluetooth device. Once this step is done,

the new device will be visible in the known devices list immediately. This list also shows

the unique address of each Bluetooth device which facilitates in identifying individual

devices with identical names.

Figure 34. Known devices page showing a list of Bluetooth device names and their unique addresses.

Page 44: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

38

4.3.1.8 Available Protocols page

The Available protocols page is a like a file browser module where the user can browse

through exercise protocols that were created previously by any Admin level user. These

exercise protocol files are read during application start and the contents are shown for

quick look in one single vertically scrollable page. The user can scroll up/ down and look

at various exercise protocol files and can decide which one to use for a specific therapy

session or decide to create a new one in the Create protocol page.

4.3.1.9 Create Protocol page

An exercise protocol is a file that specifies the parameters for an exercise session. These

parameters include the name of the for future reference, Exercise protocol author’s name

for reference, Start position of the Robo-wrist with either Extension or Flexion value in

approximate degrees and Radial or Ulnar deviation value in approximate degree value,

Range of motion specified in four directions of Extension, Flexion, Radial Deviation, and

Ulnar deviation in degree measure, Number of repetitions in Extension/ Flexion and

Radial/ Ulnar deviation direction and finally the speed of movement of the Robo-wrist

ranging from Slow, Medium and Fast. By clicking on the save button the protocol file is

saved in the phone’s memory with the name defined by the author in “Protocol name” field.

Page 45: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

39

Figure 35. Available protocol page displaying the exercise protocols stored on the device

and their contents.

Page 46: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

40

Figure 36. Create protocol page which allows users to create new exercise protocols

depending on their needs.

Page 47: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

41

4.3.1.10 Start Exercise page

The start exercise page is the module wherein the user can initiate an exercise using the

Robo-wrist. To do so, first the user selects one of the available devices from a list, and

then initiates a connection. After a successful connection is established indicated by a

prompt, the user can select any exercise protocol file from another list displayed, and send

the exercise protocol file to the device. After successful transfer of the exercise protocol

file indicated by a prompt, the user can start an exercise by pressing the Start button and

can stop the exercise by pressing the Stop button anytime. Pressing the stop button opens

up a prompt for confirmation of stopping the exercise in order to avoid accidental presses.

There is also an emergency stop button which stops the Robo-wrist from moving

immediately. In order to visualize the feedback, the user can then press the “Back” button

on the phone, go to the visualizer page; connect the same device and the graph starts

plotting immediately.

4.3.1.11 Test plot page

The test plot page is used for a visual check if the Graph plotting module is currently

working or not. This module continuously plots the phone’s accelerometer values. If this

page displays a running plot without any issues, the graph plot in the Data Visualizer page

should also work properly.

Page 48: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

42

Figure 37. Start Exercise page of the Android application displaying connect, send, Start/

stop, Emergency stop functions.

Page 49: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

43

Figure 38. Test plot page plotting 3-axis accelerometer data from the phone.

Page 50: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

44

4.3.2 Desktop PC application

The bigger component of the software application is the Desktop version. This desktop

application differs from the Android version in various aspects like design model,

communication media, number of devices that it can connect at a time, and the functional

modules.

4.3.2.1 Design model

The desktop application is designed based on split panel concept, where the application

window is split into two distinct panels. One of the panels remains static or does not

change due to user actions while the other panel is dynamic and changes according to

the user actions within the application. This design also allows for navigating between the

modules better. This data flow between modules is illustrated in figure 40.

4.3.2.2 Background processes

As with any application design, the desktop application has some background process

being performed upon application start. These are

1. Check if Bluetooth module of the computer is switched on. If not prompt the user

to do so and search for known or previously paired Bluetooth devices from the

computer’s Bluetooth device register.

2. Access the application’s folder in the computer’s memory, read all the exercise

protocol files created during the previous application use, and previously used

WiFi devices.

Page 51: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

45

Figure 39. Data flow design between modules and access control in desktop application.

Data

visualizer

module

Available

Devices

module

Available

protocols

module

Create

protocol

module

Bluetooth

Exercise

module

WiFi

exercise

module

Home Page/ main module

Multi

visualizer

module

Login module

Data

visualizer

page

Message

prompt

Guest login

Admin login

Invalid login Check

login

Bidirectional flow

Single way flow

Page 52: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

46

Figure 40. Desktop application’s panel design. The left panel is the static one, the right

panel is the dynamic one.

4.3.2.3 Modules

The desktop application is divided into modules which are initialized upon button clicks but

for the Login module which is initiated upon application start. The constituent modules are

1. Login/ access control module

2. Homepage/ Main page

3. Data visualizer

4. Known devices

5. Create new Protocol

6. Available Protocol

7. Bluetooth Exercise

8. WiFi exercise

9. Multi visualizer

Page 53: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

47

4.3.2.4 Login module

The login module serves as the first screen upon application start and this module is

responsible for access control. Based on different login details multiple levels of access to

the application can be implemented. In the current scenario, there are two user profiles/

access; guest and administrator access. The guest access limits the user to access only

the data visualizer module while the administrator access allows access to all the modules.

This module is also responsible for performing all the background process mentioned

before

Figure 41. Login window of the Desktop application.

Page 54: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

48

Figure 43. Prompt window which appear upon unsuccessful login.

Figure 43. Prompt window which appear upon successful login and implies background processes are running.

Figure 44. Prompt window which appear upon successful completion of all the background processes.

Page 55: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

49

4.3.2.5 Home page

The home page of the application is the central module for traversing through various

modules. To facilitate easy use, this module is designed as a split panel where the left

panel is static and the right panel is dynamic as shown in figure 40. What this means is

that, the left static panel remains the same throughout the application use while the right

dynamic panel changes according to user actions performed within the application. Each

button on the left static panel, when pressed shows a corresponding module screen as

indicated in the name of the button within the right dynamic panel. This design was opted

for its simplicity, and also for easy traversal between modules.

4.3.2.6 Data Visualizer

The data visualizer is the module responsible for plotting feedback data from the Robo-

wrist via the microcontroller box in real-time. In this module the user can select device he

wants to connect through a drop down list, connect, and the graph module will

automatically start plotting the feedback values if the Robo-wrist device is sending

feedback data. This module is also the only module visible to guest access profile.

Currently this module is limited to visualizing data via Bluetooth connection only. A

separate module for exclusively visualizing the WiFi connected devices is also

incorporated in the application.

Page 56: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

50

Figure 45. Data visualizer module for the desktop application with real-time graph plot of

feedback data.

4.3.2.7 Available devices module

The available devices module shows the user a list of known Bluetooth devices and WiFi

devices previously used. This list is also shown in other modules when the user tries to

connect and perform exercises or visualizations. If a new Bluetooth device is to be

connected, the user manually goes to the Computer’s Control panel, open Bluetooth

devices, search, and add the required device before the application start. This list also

shows the unique address of each Bluetooth device which facilitates in identifying

individual devices with identical names. Adding a new WiFi device is facilitated within the

Page 57: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

51

application in this module itself and is as easy as entering a name, entering the device’s

IP address and clicking on Add button.

Figure 46. Available devices module for the Desktop application.

Page 58: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

52

4.3.2.8 Available Protocols module

The Available protocols module is a like a file browser module where the user can browse

through exercise protocols that were created previously by any Admin level user. These

exercise protocol files are read during application start and displayed as a list of filenames.

To view the contents the user can select the required file and click on Open to view the

file content on the right of the list.

Figure 47. Available protocols module for the Desktop application.

Page 59: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

53

Figure 48. Create protocol module for the Desktop application.

4.3.2.9 Create Protocol module

An exercise protocol is a file that specifies the parameters for an exercise session. These

parameters include the name of the exercise protocol for future reference, Protocol

author’s name for reference, Start position of the Robo-wrist with either Extension or

Flexion value in approximate degrees and Radial or Ulnar deviation value in approximate

degree value, Range of motion specified in four directions of Extension, Flexion, Radial

Deviation, and Ulnar deviation in degree measure, Number of repetitions in Extension/

Flexion and Radial/ Ulnar deviation direction and finally the speed of movement of the

Robo-wrist ranging from Slow, Medium and Fast. By clicking on the save button the

Page 60: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

54

protocol file is saved in the computer’s memory in the user’s My Documents folder under

the folder name Robo-wrist for reference with the name defined by the author in “Protocol

name” field.

4.3.2.10 Bluetooth Exercise module

The Bluetooth exercise module is where the user can initiate an exercise using the Robo-

wrist via Bluetooth as the name suggests. To do so, first the user selects one of the

available devices from a list, and then initiates a connection. After a successful connection

is established indicated by a prompt, the user can select any exercise protocol file from

another list displayed, and send the exercise protocol file to the device. After successful

transfer of the exercise protocol file indicated by a prompt, the user can start an exercise

by pressing the Start button and can stop the exercise by pressing the Stop button

anytime. Pressing the stop button opens up a prompt for confirmation of stopping the

exercise in order to avoid accidental presses. There is also an emergency stop button

which stops the Robo-wrist from moving immediately. There is a built in visualizer within

this module that starts plotting as soon as the device is connected and sends feedback

data.

4.3.2.11 WiFi Exercise module

This module as the name suggests is exclusively built for control WiFi enabled Robo-

wrists. In this module the user can select two devices simultaneously. The right dynamic

panel is divided into two to allow for side by side control and visualization of two devices

simultaneously. To start exercise the user can select devices from a list displayed, connect

to them, send protocol files and start exercise on both the devices. Stopping the exercise

anytime is also facilitated for each device separately. Visualizations for each device

appear in the corresponding graph within each device frame and start as soon as the

devices are connected and start sending data.

Page 61: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

55

Figure 49. Bluetooth exercise module for the Desktop application.

Page 62: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

56

Figure 50. WiFi exercise module for the Desktop application with side by side frames for

two devices and graph visualizations.

Page 63: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

57

4.3.2.12 Multi Visualizer

The multi visualizer module is designed just to visualize multiple WiFi enabled Robo-

wrists. At most this module can visualize four graphs in real-time simultaneously from four

different WiFi devices. All the exclusive visualizer modules like the Data visualizer and the

Multi visualizer modules do not allow the user to control the device but passively monitor

the device.

Figure 51. Multi visualizer module with four simultaneous visualizations via WiFi.

Page 64: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

58

4.4 Arduino Control Algorithm

The Arduino microcontroller works on a ‘C’ program uploaded to it from a computer. This

is the program or code that makes the microcontroller do actions that it does. Hence the

algorithm for proper functioning of the Robo-wrist based on the software application’s

commands is very crucial to the entire Robo-wrist system. For the Robo-wrist application

the Arduino code consists of modules which do specific functions in a continuous loop

based on certain conditions. A flow chart depicting how the Arduino code works through

modules is shown in figure 53. The various modules in the code involved are

1. Initialization module

This module sets up all necessary variables, flags, and starts communication for

both Bluetooth and WiFi.

2. Communication module

This module receives data from the software application and sends feedback data

back to the application continuously. It is also responsible for processing the

incoming data into commands and exercise protocol data.

3. Motion module

This module sends the required PWM signal to the motors to move in order to

perform Extension/ Flexion and radial/ Ulnar deviation. It also checks for necessary

flags, repetitions completed, and start and stops the motor from running.

Page 65: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

59

Figure 52. Arduino control algorithm flow chart for Robo-wrist system.

Yes

Yes

Yes

Yes

Send data via

Bluetooth/ WiFi

Send data via

Bluetooth/ WiFi

Motion

module

Power On/ Initialization

Loop start

Listen for data

Stop Read,

process

data

Start

Start Flag

true?

Set Start

Flag true

Set Start

Flag false

No data

No

Do Extension/

Flexion

Reps

completed?

No

Do Radial/

Ulnar deviation

Reps

completed?

Set Start

Flag false

Bluetooth/ WiFi connect

No

No

Initialization

module

Communicat

ion module

Page 66: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

60

CHAPTER 5 CONCLUSION

The Robo-wrist system is a unique system designed to assist the physiotherapists in

providing health care to more patients simultaneously and can bridge the gap between

the increasing need for physiotherapists and the available workforce. The system is in

its infancy as major changes to ergonomics, compactness and stress/ reliability testing

are required in the future to make it into a commercial product yet the system’s three

separate component approach makes it easier to replicate and also debug in cases of

functional issues. The Software application shares this feature of easy debugging,

extensibility, and modularity which has the potential to extend its functionality to many

levels. The software application’s visual GUI based approach is not only meant to control

the Robo-wrist and visualize data but could be transformed into an analytic tool and can

be combined with different types of devices with similar functionalities. The software

application is not fully exclusive to the Robo-wrist; any other mechanical exoskeleton

with similar operational specifications can be retrofitted to the Microcontroller box and

be used with the software application. Another interesting feature of the software

application is that, it can be used as a passive visualizer with any device that transmits

feedback data via Bluetooth or WiFi. These kinds of features make the software

application much more flexible and versatile to use, reducing the amount of software

needed for multiple types of devices. As for future work, the Robo-wrist needs to be

more ergonomic, compact, the microcontroller box needs to very compact and mounted

on the exoskeleton itself and the software application needs additional analytical tools.

Page 67: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

References

1. Statcan.gc.ca, 'The 10 leading causes of death, 2011', 2014. [Online]. Available:

http://www.statcan.gc.ca/pub/82-625-x/2014001/article/11896-eng.htm. [Accessed: 04-

March- 2015].

2. A. Hakim, F. Silver and C. Hodgson, 'Organized stroke care: A new era in stroke

prevention and treatment’, Canadian Medical Association Journal, vol. 159, no. 6, p. S1,

1998.

3. Heartandstroke.ns.ca, 'Stroke Statistics-Copy', 2015. [Online]. Available:

http://www.heartandstroke.ns.ca/site/c.otJYJ7MLIqE/b.3669321/k.BD5A/Stroke_Statistic

s.htm. [Accessed: 04- March- 2015].

4. Public Health Agency of Canada, 'Tracking Heart Disease and Stroke in Canada', 2009.

5. Public Health Agency of Canada, 'Stroke Highlights', 2011.

6. Physiotherapyalberta.ca, 'Physiotherapy Alberta College + Association: The Movement

Specialists: The Basics of Stroke Rehabilitation', 2015. [Online]. Available:

http://www.physiotherapyalberta.ca/public_and_patients/the_you_movement_blog/the_b

asics_of_stroke_rehabilitation. [Accessed: 04- March- 2015].

7. M. Mahler, K. Zuger, K. Kaspar, A. Haefeli, W. Jenni, T. Leniger and J. Beer, 'A cost

analysis of the first year after stroke--Early triage and inpatient rehabilitation may reduce

long term costs', Swiss medical weekly, vol. 138, no. 31-32, pp. 459-465, 2008.

8. K. Salter, M. Hartley and N. Foley, 'Impact of early vs delayed admission to rehabilitation

on functional outcomes in persons with stroke', Journal of Rehabilitation Medicine, vol.

38, pp. 113-117, 2006.

9. Www12.statcan.gc.ca, '2006 Census of Canada: Topic-based tabulations', 2015.

[Online]. Available: https://www12.statcan.gc.ca/census-recensement/2006/dp-pd/tbt/Rp-

ENG.cfm?TABID=1&LANG=E&A=R&APATH=3&DETAIL=1&DIM=0&FL=A&FREE=0&G

C=01&GID=837928&GK=1&GRP=1&O=D&PID=92104&PRID=0&PTYPE=88971,97154

&S=0&SHOWALL=0&SUB=0&Temporal=2006&THEME=74&VID=0&VNAMEE=&VNAM

E. [Accessed: 04- March- 2015].

61

Page 68: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

10. Canadian Institute for Health Information, 'Physiotherapists in Canada 2010 National and

Jurisdictional Highlights and Profiles', 2011.

11. Wikipedia, 'Hardiman', 2015. [Online]. Available: http://en.wikipedia.org/wiki/Hardiman.

[Accessed: 04- March- 2015].

12. Lockheedmartin.ca, 'HULC · Lockheed Martin', 2015. [Online]. Available:

http://www.lockheedmartin.ca/us/products/hulc.html. [Accessed: 04- March- 2015].

13. Cyberdyne.jp, 'CYBERDYNE HAL THERAPY', 2015. [Online]. Available:

http://www.cyberdyne.jp/services/HALTherapy.html. [Accessed: 04- March- 2015].

14. Multivu.prnewswire.com, 'Raytheon unveils lighter, faster, stronger second generation

exoskeleton robotic suit', 2015. [Online]. Available:

http://multivu.prnewswire.com/mnr/raytheon/46273/. [Accessed: 04- March- 2015].

15. Intl.eksobionics.com, 'Ekso Bionics - Exoskeleton, wearable robot for people with

paralysis from SCI or stroke', 2015. [Online]. Available: http://intl.eksobionics.com/ekso.

[Accessed: 04- March- 2015].

16. Rewalk.com, 'ReWalk – Personal – ReWalk', 2015. [Online]. Available:

http://www.rewalk.com/products/rewalk-personal/. [Accessed: 04- March- 2015].

17. Bleex.me.berkeley.edu, 'BLEEX | Berkeley Robotics & Human Engineering Laboratory',

2015. [Online]. Available: http://bleex.me.berkeley.edu/research/exoskeleton/bleex/.

[Accessed: 04- March- 2015].

18. Bleex.me.berkeley.edu, 'ExoHiker™ | Berkeley Robotics & Human Engineering

Laboratory', 2015. [Online]. Available:

http://bleex.me.berkeley.edu/research/exoskeleton/exohiker/. [Accessed: 04- March-

2015].

19. Bleex.me.berkeley.edu, 'ExoClimber™ | Berkeley Robotics & Human Engineering

Laboratory', 2015. [Online]. Available:

http://bleex.me.berkeley.edu/research/exoskeleton/exoclimber/. [Accessed: 04- March-

2015].

20. Bleex.me.berkeley.edu, 'eLEGS™ | Berkeley Robotics & Human Engineering

Laboratory', 2015. [Online]. Available:

62

Page 69: Robo-Wrist Controller - Summitsummit.sfu.ca/system/files/iritems1/16486/etd9055... · 7. Single Robo-wrist control/ monitor at any given time via Bluetooth 8. Multiple Robo-wrist

http://bleex.me.berkeley.edu/research/exoskeleton/elegs%E2%84%A2/. [Accessed: 04-

March- 2015].

21. Bleex.me.berkeley.edu, 'Austin | Berkeley Robotics & Human Engineering Laboratory',

2015. [Online]. Available: http://bleex.me.berkeley.edu/research/exoskeleton/medical-

exoskeleton/. [Accessed: 04- March- 2015].

22. Bleex.me.berkeley.edu, 'HULC™ | Berkeley Robotics & Human Engineering Laboratory',

2015. [Online]. Available: http://bleex.me.berkeley.edu/research/exoskeleton/hulc/.

[Accessed: 04- March- 2015].

23. 'Titanarm-about', 2015. [Online]. Available: http://titanarm.com/about. [Accessed: 04-

March- 2015].

24. Equipoisinc.com, 'x-Ar exoskeletal arm support', 2015. [Online]. Available:

http://www.equipoisinc.com/products/xAr/. [Accessed: 04- March- 2015].

25. Jaecoorthopedic.com, 'The JAECO WREX - Wilimington Robotic Exoskeleton, WREX

MultiLink Arm', 2015. [Online]. Available:

http://jaecoorthopedic.com/products/products/WREX%3A-Wilmington-Robotic-

EXoskeleton-Arm.html. [Accessed: 04- March- 2015].

26. Saebo.com, 'SaeboFlex', 2015. [Online]. Available:

http://www.saebo.com/products/saeboflex/. [Accessed: 04- March- 2015].

27. Solidworks.com, 'SolidWorks', 2015. [Online]. Available: https://www.solidworks.com/.

[Accessed: 04- March- 2015].

63