Goal Management Training: A Web-Based Approach

by

Halla Fahmi

A thesis submitted in conformity with the requirements for the degree of Master of Science

Graduate Department of Rehabilitation Science University of Toronto

© Copyright by Halla Fahmi (2013)

ii

Goal Management Training- A Web-Based Approach

Halla Fahmi

Master of Science

Graduate Department of Rehabilitation Science

University of Toronto

2013

Abstract

This study was undertaken to introduce an innovative approach to cognitive rehabilitation

intervention (Goal Management Training- GMT) delivery, through a web-based platform

administered to adults with cerebrovascular disease (CVD) or suffering from CVD risk factors

who presented with executive function impairments. The feasibility of this approach was

investigated by developing a semi-quantitative-qualitative tool to measure therapist competence

and group engagement adapted from the Cognitive Therapy adherence-to-protocol scale. Results

from two raters analyzing random web-based GMT session recordings showed no compromise in

any aspect measured. In addition, the efficacy of the intervention was established using

neuropsychological and functional outcome measures, with significant results observed on the

Goal Attainment Scale functional measure. To our knowledge, this study is the first of its kind to

a) employ videoconferencing technology to overcome accessibility barrier to cognitive

rehabilitation and b) develop an adherence to protocol tool to measure various aspects of GMT.

iii

Acknowledgments

Firstly, I would like to sincerely thank Drs. Sandra Black and Gary Turner for giving me

the opportunity to work on this unique project under their supervision. This thesis would not

have been possible without their continued support and guidance in every step of the way. I am

especially thankful to Dr. Black for referring me to Dr. Prathiba Shammi who has extensively

helped me gain professional experience as a Psychometrist and been a wonderful teacher. I am

indebted to Dr. Gary Turner for his direction, encouragement, and above all patience in helping

me formulate and attain my goals for the past two years. I would also like to thank my committee

members Drs. Brian Levine, Deirdre Dawson and Elsa Marziali for their helpful advice, expert

feedback and continued guidance in steering this project, as well as Dr. Laura Middleton who has

helped me with her valuable expertise. This research is only possible with the support of various

funding bodies that have supported me and this project: The Heart and Stroke Foundation Centre

for Stroke Recovery, the North and East GTA Regional Stroke Network and the Department of

Rehabilitation Science, University of Toronto.

I would like to acknowledge and express my gratitude to the members of the research

team at Baycrest- Rotman Research Institute who have helped me with all aspects of the study:

Priya Kumar, Wayne Khuu and Marjorie Green. I am also grateful for the help of Dr. Marziali’s

research team: Illia Tchernikov, Martin Yeung and Malcolm Chan, for their aid and training with

the technology. I owe my deepest appreciation to the generous commitment and involvement of

the eight participants who dedicated their time to this study providing us with valuable data and

findings.

Thank you to all my friends especially Sidrah Arshad, for referring me to the Graduate

Department of Rehabilitation Science and for her mentorship all throughout. Finally, I would like

to express my sincerest words of thanks and gratitude to my family; my father Dr. Khalid Fahmi

for being the role model who has inspired me to dive into the world of scientific research, my

mother for her unconditional love and for always believing in me and my sisters for their

continued support. Not forgetting my nephew Omar Estwani for never failing to make any day a

good day. Last but not least, I thank God (Alhamdulillah) for all the blessings above.

iv

Table of Contents

Abstract ........................................................................................................................................... ii Acknowledgments .......................................................................................................................... iii

Table of Contents ........................................................................................................................... iv List of Tables ................................................................................................................................. vi List of Figures ............................................................................................................................... vii List of Appendices ....................................................................................................................... viii Chapter 1 Introduction .................................................................................................................... 1

Chapter 2 Literature Review ........................................................................................................... 3 Introduction ................................................................................................................................ 3 2.1 Frontal executive functions: neural systems ....................................................................... 4

2.2 Models of Frontal Executive Functions .............................................................................. 6 2.2.1 The Supervisory Attentional System SAS (Shallice & Norman, 1980) ................. 6 2.2.2 Working Memory: Central Executive System ........................................................ 7 2.2.3 Goal Selection and Goal Neglect (Duncan, 1986) .................................................. 8

2.3 Goal Management Training .............................................................................................. 10 2.4 Executive Dysfunction, Stroke and Cerebrovascular Disease .......................................... 12

2.5 Web-Based Delivery Platforms ........................................................................................ 12 2.5.1 E-Education ........................................................................................................... 12 2.5.2 Remote delivery of Cognitive Behavioural Therapy and cognitive

rehabilitation ......................................................................................................... 13 2.5.3 Caregiver Support Groups .................................................................................... 14

2.6 International Classification of Functioning ...................................................................... 15

2.7 Conclusion ........................................................................................................................ 18

Chapter 3 Research Plan and Protocol .......................................................................................... 19 3.1 Background ....................................................................................................................... 19

3.2 Research tools ................................................................................................................... 20 3.2.1 Goal Management Training Program ................................................................... 20 3.2.2 Web-based platform .............................................................................................. 21

Adobe connect .................................................................................................................. 21 Caring for Me Website ...................................................................................................... 22

3.3 Outcome Measures ............................................................................................................ 22

3.3.1 Adherence to Protocol ........................................................................................... 23 3.3.2 Survey ................................................................................................................... 24 3.3.3 Neuropsychological Measures .............................................................................. 24

3.3.4 Functional Outcome Measures ............................................................................. 25 3.4 Conclusion ........................................................................................................................ 27

Chapter 4 Web Goal Management Training: Online Cognitive Training .................................... 28 Chapter 5 Discussion .................................................................................................................... 45

5.1 Executive Function ........................................................................................................... 46 5.2 Efficacy of Web-Based Goal Management Training (WebGMT) ................................... 48

5.2.1 Limitations and Future Directions ........................................................................ 50 5.3 The importance of WebGMT ............................................................................................ 50 5.4 Feasibility Web-Based Goal Management Training ......................................................... 51

5.4.1 Limitations and Future Directions ........................................................................ 52 5.5 Summary and Conclusions ............................................................................................... 53

v

Appendix A: Cognitive Rehabilitation Therapy Scale for Goal Management Training –

(CRTS-GMT) ........................................................................................................................... 64 Appendix B: Participant Feedback Survey ................................................................................... 74 Appendix C: WebGMT Groups 1 and 2 compared to In-Class GMT on CRTS-GMT scale ....... 76

vi

List of Tables

Table 3.1 Goal Management Training Module Descriptions ....................................................... 20 Table 3.2 Battery Quantitative Outcome Measures ...................................................................... 26

Table 4.1 Participant Characteristics ............................................................................................ 32 Table 4.2 WebGMT Group 1 CRTS-GMT scores of two raters across randomly selected session

recordings ...................................................................................................................................... 38 Table 4.3 WebGMT Group 2 CRTS-GMT scores of two raters across randomly selected session

recordings ...................................................................................................................................... 38

Table 4.4 Participant Feedback Survey ........................................................................................ 39 Table 4.5 DKEFS raw score significance using the reliable change index .................................. 40 Table 4.6 Neuropsychological tests of executive function raw score significance using students t-

test ................................................................................................................................................. 40 Table 4.7 Dysexecutive Questionnaire raw scores using students t-test ...................................... 40 Table 4.8 Goal Attainment Scale measure .................................................................................... 41

vii

List of Figures

Figure 2.1 Flow chart to illustrate the five steps in goal management training ............................ 11 Figure 2.2 International Classification of Functioning model ...................................................... 17

Figure 2.3 Barriers cycle impeding treatment ............................................................................... 17 Figure 3.1 Screen Shot of Web-based Platform ............................................................................ 22

viii

List of Appendices

Appendix A: Cognitive Rehabilitation Therapy Scale for Goal Management Training –

(CRTS-GMT) ........................................................................................................................... 64 Appendix B: Participant Feedback Survey ................................................................................... 74

Appendix C: WebGMT Groups 1 and 2 compared to In-Class GMT on CRTS-GMT scale ....... 76

1

Chapter 1 Introduction

The main goal of this thesis was to investigate the feasibility and efficacy of delivering an

evidence-based cognitive rehabilitation intervention, namely Goal Management Training (GMT),

using a web-based platform administered to participants with executive dysfunction. GMT has

proven successful in improving cognitive abilities in patients with executive function

impairments (Levine, et al., 2007; Fish, et al., 2007; van Hooren, et al., 2007; Levine, et al.,

2011). It typically requires significant in-class, small-group attendance, placing time and travel

burdens on participants. While telephone and web-based delivery is common in Cognitive

Behavioural Therapy (CBT) and e-learning, to date, there are no published reports investigating

the use of real-time video conferencing platforms to deliver group-based cognitive rehabilitation

(Bergquist, Gehl, Holzworth, & Beaulieu, 2008; Machtmes & Asher, 2000). The web-based

GMT approach (WebGMT) presented in this thesis aims to increase accessibility to those who

are otherwise unable to receive treatment, and for the first time to deliver a group-based

cognitive rehabilitation intervention, establishing a novel foundation for the delivery of

therapeutic interventions via online platforms.

Accordingly, the feasibility of this innovative approach was established by developing a semi-

quantitative-qualitative tool to measure the adherence to GMT protocol and various aspects of

group engagement. This scale was devised from a CBT adherence-to-protocol scale (Blackburn,

et al., 2001) and was created to ensure no aspects of GMT protocol and therapist competence

were compromised in the web-based delivery. Moreover, the efficacy of the online intervention

was demonstrated using a battery of neuropsychological and functional outcome measures. Both

of these pieces of evidence supporting the WebGMT approach have the potential for significant

impact on the delivery of therapy, such as cognitive rehabilitation intervention, to a growing

population of patients in need. Our approach is the first of its kind to be delivered remotely for

group-based cognitive rehabilitation, with promising results that can greatly benefit patients

living in isolated communities, inaccessible to healthcare facilities and community services.

This thesis is organized in a manuscript format. Chapter 2 provides the background literature to

understand the theories forming the basis for GMT. Neural structures and cognitive processes

underlying executive function are also explained. In addition, the chapter provides a review on

2

the web-based delivery platform in the fields of CBT, cognitive rehabilitation, e-learning and

social work. Finally, Chapter 2 ties all aspects of this research to the International Classification

of Functioning model for rehabilitation research, providing the reader with a holistic picture,

highlighting the importance of this line of research. Chapter 3 then details the research protocol

and participant characteristics. Chapter 4 is written in a manuscript format to be submitted for

publication to Neuropsychological Rehabilitation. This chapter explicitly portrays all aspects of

the research and presents the findings and future directions. Finally, Chapter 5 serves as a wrap-

up chapter, summarizing the research undertaken and tying in previous findings and future

implications discussed in Chapter 2 and Chapter 4.

3

Chapter 2 Literature Review

Introduction

In this literature review, I provide an overview of frontal executive functions, including a brief

review of their neural basis and a survey of the most influential theoretical models characterizing

the cognitive processes and operations subsumed under the umbrella of executive functioning.

Frontal executive functions are a set of complex cognitive processes that allow an individual to

set and pursue goals and flexibly adapt to shifting contingencies (Levine & Turner, 2004). It is

an inclusive term that describes a collection of higher cognitive abilities, typically associated

with the frontal lobes and frontally-mediated systems (Levine, Turner, & Stuss, 2008). Cognitive

processes that have been associated with executive functions include planning, self-monitoring,

behavioural control, adapting to novel situations, complex problem solving, decision making and

the ability to sustain attention, as well as other skills such as maintaining continuous self-

awareness over time (Chan, Shum, Toulopoulou, & Chen, 2008; Levine, Turner, & Stuss, 2008;

Hart & Evans, 2006). Executive functions allow one to formulate goals, generate strategies,

arrange complicated actions and monitor behaviour all whilst flexibly dealing with new

information and inhibiting unwanted actions.

In this thesis project, frontal executive functions are conceptualized as those higher cognitive

functions necessary for goal-directed action. According to one of the earliest and most influential

models of executive functioning proposed by Norman and Shallice (1980) and later discussed in

this chapter, actions are guided either automatically, through established stimulus-response

behaviours, or in a controlled manner by engaging a supervisory attention system (SAS). The

SAS refers to mental operations called forward in novel situations or where contingencies have

shifted and automatic responses yield behavioural errors (Gilbert & Burgess, 2008). Supervisory

attention is critical for goal direction as it allows behaviour to progress towards a future goal

state by overcoming routinized behaviours (e.g. habits, environmentally dependent responses)

that may distract from or disrupt actions that would otherwise lead to goal attainment. Thus,

supervisory attention, or executive functioning, permits humans to transcend the here and now,

to overcome automatic and stimulus-driven responses and ultimately guides their actions

according to planned goals (Gilbert & Burgess, 2008).

4

As reviewed below, frontal executive functions are reliant on the frontal lobes and frontally-

mediated brain systems. As such, they are particularly vulnerable to insults to the frontal lobes

directly (e.g. traumatic brain injury, frontal strokes) or to white matter connections linking

frontal brain regions to posterior and subcortical brain structures (e.g. multiple sclerosis, healthy

and pathological aging including ischemic white matter disease). Given the prevalence of these

etiologies, executive function deficits are among the most common neuropsychological

disorders. Their devastating impact, specifically on real-world functional abilities, has made

them an increasingly important target for rehabilitation interventions.

Here, I describe how models of executive function (and dysfunction), particularly the model of

goal-neglect (Duncan, 1986), has informed the development of a rehabilitation intervention, Goal

Management Training – GMT, that is showing considerable promise in treating executive

dysfunction. In the following sections, I examine executive dysfunction as seen through the lens

of stroke and small vessel disease and review how GMT has been used to remediate executive

function deficits in these and other neurological populations. Finally, I discuss the practical

challenges and barriers to access for delivering an intensive cognitive rehabilitation treatment

such as GMT to an outpatient neurological population and examine how advances in internet-

based, remote delivery technologies could reduce these barriers and increase patient access.

In sum, this literature review chapter provides an overview of frontal executive function and

dysfunction and how a theory-driven intervention (GMT) is showing considerable promise in

treating executive dysfunction. From this review, I describe how internet-based technologies are

providing new opportunities to enhance access to GMT. In addition, I relate this to the

International Classification of Functioning framework, providing a large picture in describing the

necessity of this research. In the chapters to follow, I describe the results of a novel study

examining both the feasibility and efficacy of an internet-based approach to delivering GMT

using a web-based, remote delivery protocol.

2.1 Frontal executive functions: neural systems

Mapping of executive functions to specific brain regions, namely the frontal and prefrontal lobes,

has been extensively investigated and has proven to be challenging (Stuss & Alexander, 2000).

However, damage to the frontal lobes and/or its connections to other brain regions, which can

result from acquired or progressive brain disease impacting cortical, subcortical and white matter

5

brain structures, is strongly associated with diminished executive function. Specific impairments

include deficits in initiation, planning, sequencing, inhibition, mental flexibility and shifting

(Mattson & Levin, 1990; Alexander & Stuss, 2003). Efforts to map these brain changes have

allowed researchers to identify associations between behaviour and functional brain anatomy

(Stuss, 2006).

There is increasing evidence that frontal executive functions may be fractionated into four

distinct categories, each associated with functioning in different regions of the prefrontal cortex:

1) energization and regulation, mediated by the superior medial frontal cortex, 2) executive

cognitive functions, mediated by the lateral prefrontal cortex, 3) social, behaviour and emotional

self-regulation, mediated by the ventral medial prefrontal cortex and 4) metacognitive processes,

mediated by the frontal poles (Stuss, 2006). Much of this brain-behaviour mapping has emerged

from experimental neuropsychological research with brain-injured patients, and I briefly review

each of these in turn below.

Energization and activation refer to the processes associated with initializing and maintaining a

desired action (Levine, Turner, & Stuss, 2008; Stuss, 2006). Patients with damage to the superior

medial cortex demonstrate increased reaction times in timed tasks; that is, these patients take

longer in generating their response to a stimulus, and this can translate into apathy (Stuss &

Levine, 2002). The lateral prefrontal cortex, on the other hand, mediates higher executive and

cognitive functions, such as planning, monitoring, checking and inhibition (Stuss & Alexander,

2007; Stuss & Levine, 2002). These skills allow individuals to control lower-level cognitive

processes (Stuss & Levine, 2002). Behavioural self-regulatory functions localized in the ventral

medial prefrontal cortex are responsible for decision making underlined by reward processing

(Stuss, 2007). Patients with deficits in this area tend to perform poorly in unstructured situations;

their ability to regulate behaviour is hindered and unresponsive to internal goals and constraints

(Levine, 1999; Levine, Freedman, Dawson, Black, & Stuss, 1999). Finally, patients suffering

from damage to their frontal poles score poorly on experimental neuropsychological tests

focusing on humour appreciation and episodic memory (Stuss, 2007). This brain region linked to

metacognitive processes is responsible for self-reflection and awareness, as well as one’s

capability in making social judgments and showing empathy towards others (Stuss, 2007).

6

While it is important to understand that executive functions are comprised of multiple cognitive

processes, each localized to specific frontal brain regions – and each a potential target for

rehabilitation – one of the central focuses of this thesis project is on ‘executive cognition and its

role in goal-directed action’. Executive or ‘goal-directed’ control of behaviour has been the

subject of extensive study in the neuropsychological and cognitive science literature. In the next

section, we review three of the most influential theoretical models of goal-directed cognition and

examine how these models have informed the development of the rehabilitation protocol used in

the thesis research.

2.2 Models of Frontal Executive Functions

Several models of executive functions have been proposed over the past few decades, all of

which provide this field with a better understanding of symptom patterns occurring after frontal

lobe damage. These theories serve as potential frameworks with general principles that guide

rehabilitation through experimental research evidence. Here, I present three theories to provide

further understanding of the cognitive skills involved in executive function and how these may

inform approaches to remediating executive deficits. A critical distinction amongst the theories

presented here is in their conceptualization of executive functions as a unitary versus a

fractionatable, multi-process approach. Multiple-process theories are presented to demonstrate

how the different components of executive function are thought to be independently impaired in

brain disease and thus present potential specific rehabilitation targets. In contrast, this thesis

focuses mainly on Duncan’s single-process theory, whereby damage to a single cognitive

process (i.e., goal-directed action) is considered to be the cause of dysexecutive symptoms and

thus should be the central intervention target (Burgess & Robertson, 2002). Duncan’s theory of

goal neglect is emphasized here due to its direct link to the cognitive rehabilitation intervention

under study (Goal Management Training, see below).

2.2.1 The Supervisory Attentional System SAS (Shallice & Norman, 1980)

As described briefly in the introduction, Norman and Shallice (1980) modeled executive

functioning in terms of the variable attention demands necessary to control action. They

hypothesized two types of situations that trigger action: routine situations in which a response is

automatic and hence no attention is required to establish or maintain it, and novel situations

7

which require the formation of a new schema, monitoring/supervising, and rejection or

acceptance of the novel schema to achieve adaptive action. The former generates actions through

what is referred to as the contention scheduling mechanism, while the latter uses the supervisory

attentional system (SAS) (Norman & Shallice, 1980). Contention scheduling is a mechanism by

which a specific action schema is selected to produce behaviour from a number of competing

schemas. The selected schema guides action, whereas alternate competing schemas are inhibited

because of their lower activation value (Turner & Levine, 2008).

On the other hand, in novel situations the SAS is engaged. This mechanism is brought about and

activated under one of the following circumstances: if the situation is novel, if the action requires

additional planning or correction of action, and, finally, if the automatic action needs to be

suppressed (Norman & Shallice, 1980). In such situations, there are no precedent schemas that

can be used by way of contention scheduling; therefore, the SAS activates the process of creating

a new schema to result in an appropriate action. Patients with damage to areas of the frontal lobe

provide evidence supporting this model. Their behaviour shows an increase in either

distractibility or perseverations, which seem to be opposing tendencies within this model of

executive function; however, both of these deficits could result from a non-functional SAS.

Where the activation of a certain schema is much greater than that of other schemas and

therefore overrides them and causes perseveration. On the other hand, when a number of

schemas have similar activation strengths, they cause distractibility (Norman & Shallice, 1980).

Overall, this multiple-process model proposes that damage to discrete regions of the frontal lobe

will have variable impact on the operation of component processes within the SAS – e.g. poor

inhibition of maladaptive schemas, poor activation of relevant schemas, or poor action

monitoring (Shallice & Burgess, 1996). Reduced capacity of the SAS increases the likelihood of

maladaptive, routine, stimulus-bound, or off-task behaviours. Treatments to enhance executive

control functioning according to this model would therefore emphasize strengthening the SAS by

targeting specific processes (inhibition, monitoring) where deficits are most pronounced– which

ties in nicely with fractionation of processes discussed above (Stuss, 2006).

2.2.2 Working Memory: Central Executive System

In 1986, following up on his work with Hitch in 1974, Baddeley introduced a new concept of

working memory, which comprised initially three and later four components: the central

8

executive, the phonological loop, the visuospatial sketchpad and the episodic buffer. Based on

animal studies, this model demonstrates a helpful representation of executive function processes

in the human brain (Turner & Levine, 2008). The idea is that two “slave” systems, namely the

phonological loop and the visuospatial sketchpad, located in the posterior cortices, support the

central executive, located in the prefrontal cortex (Baddeley, 1986). These two slave systems act

to manipulate and store linguistic, visuospatial and visuoperceptual information. This

information is then organized and brought to bear on current actions through the operation of the

central executive (Baddeley & Hitch, 2000). The construct of the central executive draws heavily

on the earlier conceptualization of a SAS (Norman & Shallice, 1980). In this respect, the central

executive is an attentional control system, protecting the operation and contents of the slave

systems from distraction and coordinating the flexible entry and exit of information from these

systems based on current goal states. The central executive system (CES) was later fractionated

to include the episodic buffer as a bridge between the slave systems supporting online mnemonic

operation and long term memory representations (Baddeley & Hitch, 2000).

While similar to the Norman and Shallice (1980) conceptualization of the SAS, the CES in

Baddeley’s model represents a somewhat more unitary account of executive functioning. In this

model, the integrity of the central executive of the working memory system is a critical

determinant of executive function capacity. Thus, improving working memory and the operation

of the central executive more specifically (e.g. preserving the integrity of online information in

the context of distractions) would be a critical intervention strategy according to Baddeley’s

model of executive functioning.

2.2.3 Goal Selection and Goal Neglect (Duncan, 1986)

This model follows from Duncan’s observation that patients with frontal lobe damage show a

discrepancy between goals and actions, which he refers to as the theory of disorganized

behaviour. When presenting his model, Duncan (1986) begins by examining problem-solving

processes in both humans and machines to determine key features of the series of complex

processes undertaken by the healthy brain in executing higher cognitive skills. Firstly, Duncan

proposes that goal lists are an important prerequisite (e.g. the car must be parked), then he

requires action structures, which he describes as information-processing sub-components

composed of actions and mental operations in order to monitor and maintain goals to be achieved

9

(e.g. pushing the brake hard to avoid hitting an object). Once the set goal is achieved, a means-

end analysis must follow to determine to what degree the achieved goal matches the initial

desired goal (e.g. is the car parked within the parking lines?) (Duncan, 1986).

This theory overlaps somewhat with Baddeley’s working memory model as he characterizes

patients with executive dysfunction as being incapable of maintaining goals and goal lists on-line

using working memory (Burgess & Robertson, 2002). However, unlike Baddeley’s model where

working memory is the central feature of executive functioning, Duncan emphasizes the link

between intention and action as being a central feature of executive control. Indeed, he defines

executive dysfunction as “disregard of a task requirement even though it has been understood

and remembered” (Duncan & Stuss, 2000). For any behaviour to occur, relevant information

from the environment, required action plans and rules must be organized and mapped out in a

structured program that Duncan and his colleagues (2008) refer to as a task model. In their study

investigating the competing parts of a complex task, they show that the more complex a task

model is (i.e., the more facts, rules and requirements), the more likely it is that goal neglect will

be exhibited (Duncan, et al., 2008). Interestingly, neglect is sensitive to the mere presence of

additional knowledge, even when there is a direct instruction to disregard its content (Duncan, et

al., 2008). However, this is the case only when the task model is relatively new; the more

familiar and practiced the task is the less effect its complexity has on neglect (Duncan, Emslie,

Williams, Johnson, & Freer, 1996; Duncan, et al., 2008).

Duncan and his colleagues (2008) attribute this relationship between goal neglect and the

increase in number of task components to attentional mechanisms. Unlike Baddeley’s central

executive, attentional control in this model is not affected by real-time stimulus processing or by

increasing working memory load. Rather, it is directly affected by the complexity and novelty of

task instructions (Duncan, et al., 2008). Thus, goal-directed action is highly vulnerable to

increases in task requirements and in the knowledge that must be organized into a coherent task

model to guide behaviour (Duncan, Emslie, Williams, Johnson, & Freer, 1996; Duncan, et al.,

2008). This explains why goal neglect (i.e., executive dysfunction) is so obviously manifested in

complex real-world settings where constraints are few, contingencies are high and novelty and

demand for flexibility are constantly present.

10

Duncan’s theory of goal-directed action and goal neglect defines executive function as the

guidance of action by its intended outcome. Thus, goal setting and goal attainment are key

rehabilitation targets in this model. This conceptualization of executive function as linking

intention and action provides the theoretical foundation for Goal Management Training, the

executive function rehabilitation program utilized in this thesis project.

2.3 Goal Management Training

While the number of cognitive rehabilitation interventions specifically targeting executive

dysfunction have been increasing, the quality of evidence supporting the efficacy of these

interventions remains low (Levine & Turner, 2004; Rees, Marshall, Hartridge, Mackie, Weiser,

& Erabi Group, 2007; Cicerone, et al., 2011). In examining cognitive rehabilitation research

between 1971 and 2008, Cicerone and his colleagues identified metacognitive strategy training,

including self-monitoring and self-regulation, as an effective intervention technique for people

suffering from executive dysfunction following traumatic brain injury (TBI; Cicerone, et al.,

2011). However, they pointed out that the efficacy and underlying mechanisms of cognitive

rehabilitation interventions need to be further investigated (Cicerone, et al., 2011). In a

systematic review of executive function interventions following TBI, Kennedy and her

colleagues (2008) examined 15 studies and suggested that areas of future research should focus

on efficacy of specific intervention techniques, participant co-morbid characteristics and training

requirements for delivering interventions and setting goals (Kennedy, et al., 2008). Goal

Management Training (GMT) is one intervention that has demonstrated efficacy in improving

executive functions in acquired brain injury (TBI & stroke) and aging populations (Levine, et al.,

2000; Fish, et al., 2007; Levine, et al., 2011; Levine, et al., 2007).

Goal Management Training is a cognitive rehabilitation protocol developed by Robertson et al.

(1996), which introduces goal lists and divides tasks into small subtasks in an attempt to

remediate executive function in people with frontal lobe damage. Basing their intervention on

Robertson’s research (1996) and Duncan’s model of goal neglect (1986), as well as Shallice and

Norman’s SAS model (1986), Levine and his colleagues (2011) conducted studies with stroke

patients with executive function deficits. They implemented Robertson’s GMT protocol, where

nine modules are covered across a five week period. In this protocol, participants are expected to

attend a total of 20 hours of in-class instruction and commit to doing 20 hours of assigned

11

homework based on the material taught. The nine modules consist of material that teaches formal

problem-solving techniques, which are employed to define a goal, make a plan for its attainment,

and monitor progress. Drawing from Norman and Shallice’s early work (1980), GMT teaches

patients to use a stop-state-split strategy, whereby they are trained to recognize and leave the

automatic mode governing their behaviour and to re-establish executive control by stopping their

ongoing action, stating their goals, creating sub-goals and monitoring their progress towards

those goals. They are also taught to check their goals vis-à-vis their actual outcomes in a means-

ends analysis to determine whether corrective action is necessary to maintain progress towards

the desired goal state. Figure 2.1 displays the five stages of GMT taught across the nine modules

(Levine, et al., 2000; Levine, et al., 2007).

Figure 2.1 Flow chart to illustrate the five steps in goal management training (Levine, et al.,

2000)

12

2.4 Executive Dysfunction, Stroke and Cerebrovascular Disease

Stroke and other types of cerebrovascular disease (CVD) are common sequelae of vascular risk

factors such as hypertension, sedentary lifestyle, genetic factors, etc. Overt stroke occurs in

50,000 Canadians every year (Heart and Stroke Foundation, 2012). Canada has a population of

300,000 survivors living with the aftermath of overt stroke, 64% of whom suffer from cognitive

impairment, compared to 21% of stroke-free elderly (Heart and Stroke Foundation, 2012; Jin,

Legge, Ostbye, Feightner, & Hachinski, 2006). CVD and its risk factors, one of the most

reported of which is hypertension, are highly associated with cognitive decline, referred to as

vascular cognitive impairment (VCI) (Verghese, Wang, Katz, Sanders, & Lipton, 2009; Black S.

E., 2007). VCI is often characterized by executive dysfunction (Verghese, Wang, Katz, Sanders,

& Lipton, 2009; Hachinski, et al., 2006; Gorelick, et al., 2011). Executive dysfunction can also

be associated with signs of white matter small vessel disease, manifested as focal or confluent

silent lacunar infarcts on computerized tomography or magnetic resonance imaging (Black, Gao,

& Bilbao, 2009). Critically, small vessel or ischemic white matter disease is estimated to be 10-

20 times as prevalent as large-vessel stroke (Vermeer, Prins, den Heijer, Hofman, Koudstaal, &

Breteler, 2003; Black S. E., 2007), suggesting VCI may be an emerging public health challenge.

Given the prevalence of executive dysfunction in this population, access to evidence-based

rehabilitation treatments such as GMT is extremely important. However, as described in the

GMT protocol section above, the intervention places significant burdens on participants in terms

of in-class time along with concomitant travel burdens and related costs. Indeed, for populations

suffering from brain disease and cognitive impairment, such travel coordination and costs may

be prohibitive. To address these issues, the goal of this thesis project was to improve access to

this treatment option through the use of innovative remote delivery technology.

2.5 Web-Based Delivery Platforms

2.5.1 E-Education

The advantages of remote delivery technologies have not yet been fully realized in the delivery

of group-based cognitive rehabilitation, even though technical progress has made such distance

learning and rehabilitation opportunities possible. Using video-conferencing as an alternative to

traditional classroom instruction has become more widespread in corporations and universities

(Moore & Kearsley, 1996; Webster & Hackley, 1997; Sugrue, 2003). A meta-analysis of distant

13

learning techniques identified three instructional features that impact distant learning levels by

students (Machtmes & Asher, 2000).These three features were interaction available during e-

learning delivery, type of course delivered and the type of remote site. Two-way communication

provided by video-conferencing platforms allows the highest amount of interaction between

instructor and student, resulting in greater student achievement. In addition, the type of course

and type of remote site greatly impact student motivation. For instance, a course required by an

employer and delivered in a workplace environment showed the highest effect size; learners

were motivated to do well because their company was covering the expense, which in turn

affected their learning outcomes (Machtmes & Asher, 2000). Brown and his colleagues (2005)

also identified two effects on training outcomes: class size and instructor immediacy. Instructor

immediacy refers to the communication style of the instructor; the more engaging the interaction

between the instructor and students, the higher the learning (Brown, Rietz, & Sugrue, 2005). The

recommendations made to improve learning and motivation were to boost interaction and for the

instructor and students to get to know one another outside of class time (Brown, Rietz, & Sugrue,

2005). It has also been demonstrated in prior research that when the number of attendees

increases in video-conferencing classes , the students feel disconnected, therefore negatively

impacting the learning experience (Gowan & Downs, 1994).

2.5.2 Remote delivery of Cognitive Behavioural Therapy and cognitive rehabilitation

Cognitive behavioural therapy (CBT) as well as cognitive rehabilitation have used several

platforms of remote delivery such as telepractice, off-line web-based and, in recent years, real-

time videoconferencing delivery modes (Bouchard, et al., 2004; Ruwaard, Broeksteeg,

Schrieken, Emmelkamp, & Lange, 2010; Ruwaard, et al., 2009; Bergquist, Gehl, Holzworth, &

Beaulieu, 2008; Frueh, Monnier, Yim, Grubaugh, Hamner, & Knapp, 2007; Germain, Marchand,

Bouchard, Drouin, & Guay, 2009; Morland, Pierce, & Wong, 2004; Man, Soong, Hui Chan, &

Hui Chan, 2006; Arundine, Bradbury, Dupuis, Dawson, Ruttan, & Green, 2012). CBT is a

psychotherapeutic approach to solving maladaptive behaviour in which the person’s thoughts or

cognitive patterns are targeted in order to change behaviour (Alford & Beck, 1997).The therapist

works with the client as a pair to overcome the client’s problems. Studies comparing the

administration of CBT face-to-face and via a web-based videoconferencing platform with

various patients suffering from post-traumatic stress disorder (Frueh, Monnier, Yim, Grubaugh,

14

Hamner, & Knapp, 2007; Germain, Marchand, Bouchard, Drouin, & Guay, 2009; Morland,

Pierce, & Wong, 2004) and panic attacks (Bouchard, et al., 2004) demonstrated no differences in

therapeutic outcomes between these two delivery modes. For instance, Bouchard et al., 2004

compared a group of patients receiving CBT for panic attacks caused by agoraphobia in either a

face-to-face traditional method of delivery or in a videoconferencing delivery mode. The results

showed significant improvements in panic symptoms, panic-related characteristics, agoraphobic

avoidance, general anxiety, depressive affect, and general functioning (Bouchard, et al., 2004).

In their study, Germain et al., (2009) administered telepsychotherapy and face-to-face CBT to

patients with post-traumatic stress disorder and found improvements in both groups. They also

found via administered questionnaires that technical problems, the initial perception of therapy

by videoconference and the sense of presence felt during videoconferencing had no impact on

treatment outcome (Germain, Marchand, Bouchard, Drouin, & Guay, 2009). Another form of

therapy that has also utilized this novel technology is cognitive rehabilitation, where various

modes of delivery (telepractice, traditional in-class and web-based modes) of problem-solving

skill training were used on acquired brain injury patients. Results showed significance in efficacy

of treatment regardless of the mode of delivery across all groups (Man, Soong, Hui Chan, & Hui

Chan, 2006). All of these studies dealt with single therapist-patient interactions, whether face-to-

face or via on-line psychotherapy. In other words, none of them investigated the efficacy of

therapy in group settings, which is the goal of this study.

2.5.3 Caregiver Support Groups

Marziali and colleagues (Marziali, Damianakis, & Donahue, 2006) have demonstrated the

efficacy of therapeutic group interventions in caregivers using an online video-conferencing

delivery method. These studies have compared web-based caregiver groups and traditional face-

to-face psychosocial groups using various methods of qualitative analysis (Marziali & Donahue,

2006; Marziali, 2006; Marziali, Damianakis, & Donahue, 2006; Marziali, 2008; Damianakis,

Climans, & Marziali, 2008). Results show that older adults can be trained to effectively use a

computer and access websites leading to videoconferencing platforms (Marziali, 2009). In

follow-up interviews, participants reported high levels of involvement in the web-based groups

(Marziali, 2008), strong bonding between group members and group members and facilitators

(Marziali, 2006), as well as comfort and preference for using videoconferencing platforms

(Marziali, 2009). Given that in some studies up to 50% of the participants had never used a

15

computer, participants in one study were asked about the difficulty of using a computer after

sufficient training, 78% reported that the website was easy and 95% responded with positive

feedback to online meetings; after a six month follow-up, 90% of caregiver group members

reported extremely or very positive benefits from their online support groups (Marziali &

Donahue, 2006). In addition, one study collected therapist feedback in response to using

videoconferencing platforms for meeting with clients. The results showed that therapists needed

to reflect on their subjective responses to transitioning intervention skills from a face-to-face

environment to online video conferencing with a group of clients. For some, several iterations of

working online were needed to gain maximum comfort levels for using technology to

communicate with clients (Damianakis, Climans, & Marziali, 2008; Marziali, Damianakis, &

Donahue, 2006). With such robust findings, this thesis draws on the caregiver support group

success, as well as the CBT individual delivery successes, and uses online platforms to

investigate the efficacy and effectiveness of delivering GMT by remote real-time video-

conference.

2.6 International Classification of Functioning

The following section uses the International Classification of Functioning (ICF) framework to

support the project’s approach to using technology to deliver a cognitive rehabilitation program

(WHO, 2001). Rehabilitation is a continuous process that entails classification of a disease and

related factors in the environment, followed by successful administration of rehabilitation

interventions (Stucki, Ewert, & Cieza, 2002). This thesis project is aimed at focusing on the

selection of successful methods of delivery, appropriate to the given disease and environment. It

has potential to significantly improve access to cognitive rehabilitation for clients facing travel,

time, financial, mobility or other barriers that restrict in-person participation in cognitive

rehabilitation therapy, making it an easily translated research program.

Within the rehabilitation field, it is essential to have an organized framework that can be utilized

in research and clinical practices. For this reason, the World Health Organization created a

standard framework called the International Classification of Functioning, Disability and Health

(ICF) (Stucki, Ewert, & Cieza, 2002), which provides a standard language for all health-related

research, from both individual and societal perspectives (Vrankrijker, 2003). It incorporates the

health condition, which reflects body functions and structures, in relation to daily activities and

16

participation in society (Vrankrijker, 2003). It also considers the environmental and personal

factors that may play an important role in disability. The research I am conducting fits within the

ICF framework (shown in Figure 2.2) on several levels.

The approach (i.e., web-based delivery) takes into consideration health, environmental and

personal factors and as such attempts to bridge gaps among these three factors to decrease

barriers that typically reduce access to people requiring therapy. For example, a stroke patient

suffering from executive dysfunction, within the ICF, has a cognitive impairment arising from

brain network dysfunction, which in turn interferes with goal-directed activities and contributes

to restriction of successful participation in the society. The result of the impairment and

restrictions posed through contextual factors affect the ability to comply with treatment to

compensate or restore function arising from the brain pathway, creating a cycle that is difficult to

break (see Figure 2.3). In other words, decreased access to the treatment of the cognitive

impairment in this case further exacerbates the impairment. By providing alternative means in

accessing treatment, in this case cognitive rehabilitation, we are overcoming barriers arising from

the patient’s impairment and pre-existing environmental obstacles (such as geographical barriers)

to work around the patient’s personal circumstance in order to provide rehabilitation.

This model can also be beneficial in guiding the selection of outcome measures for analyzing

efficacy of treatment. For instance, functional outcome measures (discussed in Chapter 3), which

assess everyday functions and examine the participation aspect of a disability, shed light on the

direct benefits an intervention has on the individual’s life. On the other hand, neuropsychological

measures examine the effects of an impairment of body functions and structure on activity,

providing us with a method of assessing limitations placed on the individual (Bilbao, et al.,

2003).

Moreover, specific GMT material incorporates all branches of this model. Each module

(described in Chapter 3) introduces and teaches concepts explaining the health condition (i.e.,

executive dysfunction) and its effects on activities and participation (e.g. failure to plan a

vacation), and also draws on examples from the participants’ lives, incorporating personal and

environmental factors that may benefit in the training. In conclusion, the ICF model serves as a

point of reference in which the interaction of various aspects of a disability are analyzed and

whereby rehabilitation is tailored accordingly.

17

Figure 2.2 International Classification of Functioning model (WHO, 2001)

Figure 2.3 Barriers cycle impeding treatment

18

2.7 Conclusion

This chapter reviewed the literature on executive function, executive dysfunction in the stroke

population, and Goal Management Training, as well as provided the background on the research

done in delivering various therapeutic and educational materials through the World Wide Web.

The rationale behind using the ICF, an international model of rehabilitation, for this project has

been discussed. The next several chapters describe the specific aspects of my Master’s research

(Chapter 3- Methods, Chapter 4- Manuscript format of study results and Chapter 5- Discussion)

which investigates remote delivery of GMT to remediate executive function deficits in a stroke

and small vessel disease population.

19

Chapter 3 Research Plan and Protocol

3.1 Background

GMT is an evidence-based cognitive rehabilitation program aimed at training cognitive

processes involved in executive functions (Levine, et al., 2007). As mentioned in Chapter 2,

GMT participation involves a significant commitment, both in time and travel demands. This

level of commitment creates potential barriers to participation, particularly for those who live in

more remote settings where transit options are limited and travel times might be prohibitive for

attendance at twice weekly group meetings. Moreover, if participants are confronting any

physical disability that might limit independent movement, time and travel burdens would be

exponentially greater.

In order to address the accessibility issue linked to patient participation in this specific cognitive

rehabilitation intervention, we have adopted the on-line novel delivery platform used in “Caring

for Me” psychosocial groups (Marziali, 2006; Marziali & Donahue, 2006; Marziali, Damianakis,

& Donahue, 2006). “Caring for Me” groups have proven successful in connecting caregivers

regardless of location. The experience was deemed highly engaging for individuals who are

otherwise unable to meet other caregivers and therapists (Marziali, 2008). While

psychotherapeutic interventions (e.g. cognitive behavioral therapy- CBT) are increasingly taking

advantage of remote delivery methods, for the most part, these have consisted of one-on-one

therapy sessions or web-facilitated interactions not occurring in real time (Bouchard, et al.,

2004). To date, with the exception of Marziali’s (2006-2008), there are no reports in the

published literature demonstrating the use of web-based methods to facilitate real-time, group

therapy interventions. Here, I describe a research project to investigate whether GMT can be

delivered via the web without compromising therapeutic efficacy. Specifically the study asks two

core questions: (i) Can GMT be delivered effectively over the web – i.e., is it feasible? (ii) Does

web-delivery impact the therapeutic efficacy of the intervention?

20

3.2 Research tools

3.2.1 Goal Management Training Program

GMT is facilitated by a psychologist or an occupational therapist specifically trained to

administer the intervention using a structured manual (Levine, Manly, & Robertson, 2008). It is

typically carried out through a five week time period in which a maximum of four participants

meet with the facilitator twice a week for two hours to complete a total of 20 hours of training

material. In addition to the in-class therapy, the program requires 20 hours of at-home assigned

work. Every two hour session covers various concepts, strategies, and real life examples to help

participants in recognizing and developing solutions to their day-to-day problems of attention,

problem solving and goal achievement (Table 3.1). Packages consisting of GMT materials that

are required throughout the training program were provided to all participants prior to start of

intervention.

Table 3.1 Goal Management Training Module Descriptions

Module Concepts & Strategies

1: The absent mind, the present mind Introduction of goal hierarchies, the mental

laboratory, absentmindedness and presentmindedness

2: Absentminded slip-ups Relation of absentmindedness to other abilities,

consequences of slips, conditions for slips how Goal

Management Training will reduce slips.

3: The automatic pilot The automatic pilot and how it leads to errors in

every day functioning

4: The automatic pilot Training to stop the automatic pilot

5: The Mental Blackboard Mental blackboard (i.e., working memory)

6: State your goal Getting side tracked from your goal (i.e., distractions)

7: Making Decisions Goal conflict and decision making

8: Splitting Tasks into Subtasks Dealing with overwhelming tasks by splitting them

into smaller tasks

9: Checking (“STOP!”) Checking (reducing slip-ups)

10: Wrap-up and feedback Review of material and taking up ongoing goals

Throughout the GMT program, participants are presented with module content explaining the

concepts and strategies mentioned above, encouraging them to share real life experiences and

21

perform interactive tasks. During a single session, various homework assignments are also

assigned. For instance, session two covers concepts such as absentmindedness in relation to other

abilities, consequences of slips, conditions for slips, and how GMT will reduce slips. In covering

this module, a task is performed where participants are presented with fruit names on the screen

appearing one at a time. After the participants see the fruit name, they are requested to clap

unless they see the word ‘pear’. This is done to demonstrate how the automatic pilot works and

how slips occur. In addition to this activity, participants are provided with worksheets (included

within their package) to record everyday slips that happen during their week before the next

session. This is done to encourage participants to recognize when they are absentminded and to

self-assess the causes of their behaviour. Therefore, each module presents concepts and covers

strategies, involves homework assignments as well as engages all participants in interactive tasks

or group discussions (see Appendix B for module outlines).

3.2.2 Web-based platform

Adobe connect

Adobe connect is an online meeting platform used to connect multiple users through a real-time

video conference. It has numerous features, including video sharing, audio sharing, screen

sharing, slideshow presentations, single or multiple presenter options, chatting and notepad note

taking options as well as recording capabilities. As many as six users may join a meeting with no

lagging problems. Administrators purchase the license to use this platform and may then create

password protected user accounts for members to join online meetings. All recorded sessions are

then uploaded onto a secure server where only administrators can access video files. Below is a

meeting screen shot labeled with all features used in our study (Fig. 3.1).

22

Figure 3.1 Screen Shot of Web-based Platform

Caring for Me Website

In order to schedule and connect users to Adobe Connect, the Caring for Me website, adapted

from Dr. Marzialis’s work on caregiver support group studies, was used. This website acts as a

gateway between users and the meeting. Once the users are logged on, they have access to a

plethora of resources, from logging their food intake to accessing specific links provided to them

by their therapist. In this study, however, the website was strictly used to schedule and attend

meetings. To insure security of the web site users and content, the web site is accessible only

through the use of unique passwords for each user. Consequently the web site is not available in

the public Internet domain. Within the web site there are three levels of security; the site

administrator has access to all links and user groups, the professional facilitators have access to

other facilitators but only to the members within the groups that they facilitate, and the

participants have access only to the members within their group. All e-mail messages and video

conferencing sessions are encrypted and stored on a secure server for subsequent analysis.

3.3 Outcome Measures

The objectives of this research were a) to determine whether GMT can be delivered using the

web-based platform described above (WebGMT) and b) whether WebGMT is as effective on-

line as it has been in class (Levine, et al., 2011). To address each of these questions, two

approaches to collecting data were carried out; Approach 1- focusing on the feasibility of the

23

WebGMT, determined using qualitative (semi-quantitative) measures, and Approach 2- focusing

on the efficacy of the web-based intervention, determined through quantitative

neuropsychological outcome measures, self-report measures and functional outcome measures.

Qualitative Measures

Two main tools were used to qualitatively analyze the web-based delivery mode: an adherence-

to-protocol scale and a survey collected from all participants reflecting their learning experience

using the technology.

3.3.1 Adherence to Protocol

The adherence-to-protocol scale was adapted from an existing revised cognitive behavioral

therapy scale (CTS-R) originally devised to assess competence in cognitive therapy (Blackburn,

et al., 2001). The CTS-R consists of 12 items, some of which are labeled general items as they

are not CBT-specific, such as feedback, collaboration, efficient use of time and interpersonal

effectiveness. Other items are labeled cognitive therapy-specific items, such as agenda setting

and adherence, eliciting appropriate emotional expression, eliciting key cognitions and more. For

the purposes of our study, and in order to assess GMT specific techniques, the CBT specific

items were either eliminated or revised and GMT theoretic bases were incorporated in the items

added. The scale was transformed from a 12-item CBT scale to a 9-item GMT scale. As with the

CTS-R, each item is rated across a six-level model of competence adapted from the Dreyfus

level of competence model using a seven-point Likert scale ranging from 0 - where the therapist

did not adhere to that aspect of therapy to 6- where there is adherence at very high skill (Dreyfus,

1989). For scoring, the maximum score on the CTS-R is 72 (that is, 12 items receiving a score of

6). To determine a minimum level of competence, the Newcastle Cognitive Therapy Centre set

the standard of 36, making it an average of three marks per item (James , Blackburn, & Reichelt,

2001). Following the same logic, for the devised 9-item scale of GMT, the maximum score is 54

and the minimum level of competence is 27, for an average of three marks per item.

The CTS-R scale with the above mentioned revisions was then named The Cognitive

Rehabilitation Therapy Scale for Goal Management Training (CRTS-GMT) and was completed

by two raters in a sample of the sessions, specifically three to five out of the 10 recorded

24

sessions, of the web-based delivery program. Attached in Appendix C is the complete CRTS-

GMT with specific features for each item.

3.3.2 Survey

The short survey created acts as a self-report tool devised to tap into participant experience using

the online technology. The survey consists of four closed-ended questions asking about the

participants level of comfort with the technology, their benefit from the technical support

provided, their opinion on how helpful the web-based platform was in the learning experience, as

well as whether given the choice in the future they would prefer an online to an in-class

participation. In addition, we included an open-ended feedback section where participants

provided any other comments they had on the overall experience. This qualitative tool allows us

to gather data reflecting how useful the participants found the web-based delivery mode as well

as suggested changes that can be implemented in the future. Attached in Appendix B is the short

survey described above.

Quantitative Measures

3.3.3 Neuropsychological Measures

The efficacy of treatment was assessed via the administration of these tests immediately before

intervention and immediately after for pre and post cognitive changes. All assessments were

administered by a trained psychometrist.

The neuropsychological battery included standardized tests such as trail making A and B to

assess visual attention and task switching, digit span for attention and verbal working memory

(Wechsler, 1985), auditory consonant trigrams for verbal working memory and divided attention

(Brown, 1958) as well as self-ordered pointing task for planning and working memory (West,

Ergis, Winocur, & Saint-Cyr, 1998). To assess frontal cognitive processes such as planning,

problem solving, learning and maintenance of instructional sets, two versions of the tower task

were used in this study to eliminate practice effects by alternating items from the original task

(Delis, Kaplan, & Kramer, 2001; Levine, et al., 2011). Moreover, some tests from the Delis-

Kaplan Executive Function System (DKEFS) were used: tests to assess verbal fluency,

examining frontal and temporal regions; design fluency tests to assess fluency in the spatial

domain; and an additional measure of attention, inhibition and switching using the Stroop test

25

(Delis, Kaplan, & Kramer, 2001). All the neuropsychological tests were analyzed using a paired

t-test of raw scores. In addition, the DKEFS tests were analyzed using an extension to the

reliable change index (RCI). The RCI is defined as the change in a client’s score divided by the

standard error of the difference for the tests, yielding a standardized score representing the

change observed (Jacobson & Truax, 1991). Additionally, the extension to the RCI takes into

account the practice effects (i.e., improvement in a test solely due previous exposure and not the

intervention) by incorporating a constant based on the group-level average change (Parsons,

Notebaert, Shields, & Guskiewicz, 2009), yielding an RCI “corrected” for practice effects

(Heaton, et al., 2001). The DKEFS tests are the only measures in which the reliability values

were available and thus were the only ones analyzed using the RCI.

3.3.4 Functional Outcome Measures

Self-reported changes were determined using questionnaires collected from patients and their

significant others. The Dysexecutive Questionnaire (DEX) was included as a 20-item

questionnaire designed to assess changes in inhibition, intention, personality, cognition and

behavioural changes (Burgess, Alderman, Evans, & Emslie, 1996). A patient-specific outcome

measure known as goal attainment scaling (GAS) was used as a measure of clinically important

change in cognitive rehabilitation (Rockwood, Joyce, & Stolee, 1997). For the GAS measure,

each of the four participants of the second WebGMT group met with the psychometrist in the

pre-assessment stage to formulate a minimum of one goal that was individually identified to suit

their level of performance. These goals were set such that the participants worked towards them

throughout the five-week training session independent of the GMT content. For each goal, the

psychometrist along with the participant determined level of achievement across a 5-point scale

ranging from -2 to +2. At the end of this session, the goals were reviewed with the participant

again and they were asked to work on them for the next five weeks. At the post-assessment

session the achievement level was determined, if the participant reported expected level of

outcome, they were give a score of 0, if they reported a somewhat less level of outcome, they

were given a score of -1 and if they reported much less level of outcome, they received a score of

-2. Consequently, if the participant reported a somewhat more than expected level of outcome,

they received a score of +1 and if they reported a much more than expected level of outcome,

they received a score of +2. Table 3.2 lists all outcome measures and provides a brief description

of each.

26

Table 3.2 Battery Quantitative Outcome Measures

Measure Assessment Domain Description

Neuropsychological

Assessments

Trails A Visual attention Test has 25 scattered dots on a piece of

paper. The dots have numbers and subjects

are asked to connect the numbers in

sequence as accurately and quickly as they

can.

Trails B Visual attention and

task switching

The second part to Trails A. Trails B has 25

dots with numbers and letters scattered. The

subject is instructed to alternate between a

number and a letter connecting them in

sequence as quickly and accurately as they

can.

Consonant

Trigrams

visual search speed,

scanning, speed of

processing, mental

flexibility, and task

switching

Subjects listen to a string of three

consonants immediately followed by a

number. The subject is asked to count

backwards in threes until instructed to recall

the consonant trigram.

Verbal Fluency Verbal fluency (frontal

and temporal regions)

Phonemic Fluency: subjects are instructed to

say as many words as they can that begin

with specific letters within a given time (i.e.,

as many words as you can beginning with

the letter ‘c’).

Semantic Fluency: subjects are instructed to

say as many words from a category as they

can within a given time (i.e., as many

animals as you can name within 60 seconds)

Design Fluency Verbal fluency in

spatial domain

Subjects are provided with a paper with

boxes. Each box contains dots. Subjects are

instructed to connect four dots in each box

making each line touch at least one other

line at a dot. Subjects are instructed to make

those designs working as quickly as they can

and by not making the same design more

than once. In other parts subjects are also

provided with distractor dots and asked to

ignore them or alternate between filled and

empty dots.

Stroop Inhibition and attention Subjects are presented with names of

colours printed in a different ink colour.

Subjects are asked to name the colour of the

ink the word is printed in and not the name

denoted by the word.

Tests of Everyday

Attention

Selective attention,

sustained attention,

and attentional

switching

Subjects are asked to perform a task that is

based on an everyday scenario. They are

required to count floor numbers in an

elevator using auditory and visual stimuli.

They are required to count low, medium and

high pitch tones to determine what floor the

elevator stops at.

Tower Task Planning, problem

solving, learning, and

maintenance of

instructional sets

Subjects are presented with three rods, and

disks that can be stacked on each rod

creating a tower. Subjects are also shown a

picture of the a tower that they must create

following three rules, by moving only one

disk at a time, by not placing a small piece

27

on a larger piece, and by using only one

hand. They are instructed to work as quickly

as they can with the fewest moves possible.

Self-Ordered

Pointing Task

Planning and working

memory

A set of stimuli are presented on a number

of sheets in a booklet. The subjects are

instructed to point to one stimulus at a time,

without ever pointing at the same stimulus

twice in each set.

Functional

Outcome Measures

Dysexecutive

Questionnaire

(DEX)- Self

Self-report of every

day executive

problems

Subjects are asked to answer questions

related to everyday executive function

problems rating each item from 0-never to 4-

very often.

DEX- Other Other report of

everyday executive

problems

Subjects significant others are requested to

complete the other form of the Self-DEX

questionnaire.

Goal Attainment

Scale (GAS)

Goal achievement A functional outcome measure in which

goals are set and revisited post-intervention

to determine achievement level.

3.4 Conclusion

To the best of our knowledge, this is the first time that a qualitative tool has been devised to

assess levels of competence in administering GMT. The adherence-to-protocol measure (CRTS-

GMT), in combination with efficacy outcome measures assessing the novel delivery of GMT

(WebGMT), makes this both a comprehensive and innovative approach to cognitive

rehabilitation.

28

Chapter 4 Web Goal Management Training: Online Cognitive Training

(In preparation for submission to Neuropsychological Rehabilitation)

Authors: Fahmi, H., Middleton, L., Marziali, E., Dawson, D., Levine, B., Turner, G., & Black.,

S.E.

Abstract

The aim of this study was to evaluate the feasibility and efficacy of an innovative, web-based,

cognitive rehabilitation intervention (Goal Management Training - GMT) for adults with

cerebrovascular disease (CVD) or suffering from risk factors of CVD and reporting executive

function complaints (N=8). GMT is one of the few evidence-based interventions for remediating

executive control dysfunction and is typically delivered using a small-group, in-class format.

Here we report the results of a pilot study to investigate the feasibility and efficacy of a remote,

web-based GMT delivery protocol to reduce barriers of accessibility. To assess feasibility of web

delivery, a rating scale was adapted and further developed to analyze various aspects of

intervention delivery such as alignment with the training protocol, therapist competence and

participant engagement. Qualitative analyses of recorded training sessions revealed high protocol

adherence, competence and engagement. Self-reported feedback questionnaires showed positive

opinions of technology used within the learning experience. To assess intervention efficacy, a

battery of neuropsychological measures was administered at baseline and post-intervention. No

significant results were observed on standardized neuropsychological measures; however,

improvements were observed on the Goal Attainment Scale, a functional outcome measure of

goal attainment in real world settings and a critical measure of GMT intervention success. This

pilot intervention study is the first to demonstrate both feasibility and efficacy of a remote, web-

based platform for delivering small-group cognitive rehabilitation. These results suggest that

web-based cognitive rehabilitation offers considerable promise as a tool for improving access to

therapy for persons facing significant physical, geographic or economic barriers to accessing

cognitive rehabilitation.

29

Introduction

Rehabilitation of executive control dysfunction: Goal Management Training

Cognitive dysfunction secondary to frontal lobe damage or damage to frontally-mediated

systems typically presents in the clinic as executive control problems with deficits in areas such

as planning, problem solving, behavioural control, mental flexibility, decision making and the

ability to sustain attention (Levine, Turner, & Stuss, 2008). According to Duncan and Miller

(2002), these executive control functions are primarily associated with flexible deployment of

attentional resources necessary to complete a given task – i.e., guidance of goal-directed

behaviour. Sustained attention is especially crucial to goal-direction in non-routine situations

requiring the persistent maintenance of higher order goals in working memory with competing

goals; environmental distractors may lead to off-task behaviours (Levine, et al., 2011). Levine

and colleagues (2011) describe this phenomenon of goal neglect as the displacement of a novel

goal in working memory due to the passive activation of a habitual response. In remediating

these functions, an evidence-based cognitive rehabilitation intervention, Goal Management

Training (GMT) (Levine, Manly and Robertson, 2008), has been developed.

GMT is a strategy-based intervention that promotes the re-engagement of endogenous sustained

attention in patients (or healthy older adults – see van Hooren et al., 2007; Levine et al., 2007)

who have been diagnosed with executive function deficits or who express difficulties in goal

direction in their everyday lives. The program emerges from Duncan’s model of goal neglect

(1986) which conceptualizes executive control dysfunction as the separation of ongoing action

from its original intent. The intervention promotes meta-cognitive strategies to aid participants in

‘stopping’ off-task behaviours, refocusing attention towards relevant goals, creating an explicit

goal hierarchy, dividing complex tasks into small subtasks, and actively monitoring progress

towards goal attainment (Duncan, Emslie, Williams, Johnson, & Freer, 1996). Levine and

colleagues (2000) originally tested this approach in a cohort of traumatic brain injury (TBI)

patients with executive control deficits. Their results showed a significant improvement in

executive control on simulated real-life tasks. A follow-up case study involving a post-

encephalitis patient with severe executive function deficits showed similarly promising results

with evidence that the benefits of GMT transferred beyond the clinic to improvement in

everyday life and application of learned strategies to untrained situations.

30

Building upon these preliminary results, Levine et al. (2007) further investigated the

effectiveness of GMT by using an elaborated multi-session form of the intervention on normal

aging individuals who reported problems with executive functions. Following GMT, older adults

demonstrated significant improvement on simulated real life tasks as well as examiner-rated

performance on complex tasks (Levine et al., 2007). However, in this first large randomized

control trial, GMT was embedded in a larger multi-model intervention protocol, which limited

the ability of the researchers to attribute training gains to GMT alone. In a recent randomized

trial of GMT in an acquired brain injury cohort, Levine and colleagues (2011) reported training-

related gains in both standardized measures of executive control and in real-world outcomes. In

sum, evidence for the efficacy of GMT as an executive function rehabilitation intervention has

now been demonstrated in several populations including traumatic brain injury and stroke (Fish,

et al., 2007; Levine, et al., 2011; Chen , et al., 2011), normal aging (Levine, et al., 2007; van

Hooren, et al., 2007) and case studies of patients with focal cerebellar damage (Schweizer, et al.,

2008).

Web-based interventions: a novel approach to delivering GMT in the clinic

There is increasing evidence that GMT can be used to remediate executive control deficits in

populations where such deficits are clinically important. Based on the success of these efficacy

studies, efforts are now underway to expand the availability of GMT as a rehabilitation tool in

the clinic. A critical challenge going forward is how to most effectively deliver this intervention

to participants who often confront significant access barriers. In its typical implementation

format, GMT is delivered in a small group consisting of three to four participants and a trained

facilitator. Sessions are typically two hours in duration, occurring twice or once per week for five

or ten weeks (20 hours of total in-class time). Participants are also asked to complete an

additional 20 hours of at-home assignments. This schedule places significant time and travel

burdens on participants. In an urban setting, where all efficacy trials published to date have been

undertaken, these burdens include parking costs, transit access as well as safety issues. Outside

of urban centers, lack of trained facilitators or critical mass of participants in a single location,

effectively preclude GMT as a rehabilitation option. To eliminate these barriers to access, we

describe here a novel approach for delivering GMT using a web-based cognitive intervention

platform.

31

There are no published reports investigating the feasibility or efficacy of web-based delivery for

small-group cognitive therapy. However, there is considerable support for such an approach

emerging from studies of cognitive behavioural therapy (CBT). Early evidence suggests that

clinical results from web-based delivery methods closely replicate those from more standard in-

person delivery methods in panic and post-traumatic stress disorder (Bouchard, et al., 2004;

Germain, Marchand, Bouchard, Drouin, & Guay, 2009; Frueh, Monnier, Yim, Grubaugh,

Hamner, & Knapp, 2007; Morland, Pierce, & Wong, 2004). However, in these reports one-on-

one therapeutic trials were employed and group-based therapeutic methods were not assessed.

Web-based group interventions have recently been investigated in psychosocial support groups

for dementia patient caregivers (Marziali & Donahue, 2006). Through qualitative methods of

analysis, web-based psychosocial groups were evaluated and showed positive therapeutic

outcomes in areas such as group bonding, shared-experiences and extended use of technology for

building bonds amongst group members (Marziali, 2006). Critically, social workers facilitating

on-line group treatments reported comparable group bonding between clients as well as clients

and therapists (Damianakis, Climans, & Marziali, 2008).

While no studies have directly examined the efficacy and effectiveness of delivering small group

cognitive therapy using a web-based platform, evidence emerging from CBT and caregiver

groups suggest that such an approach may be feasibly undertaken without compromising

therapeutic efficacy. In this preliminary rehabilitation probe, we report the first feasibility and

efficacy data of web-based GMT (WebGMT). There are two primary research objectives. First,

we will examine the clinical feasibility of WebGMT, including assessment of protocol

adherence, using a semi-quantitative-qualitative rating scale developed for this study as well as

self-report client satisfaction surveys. Second, we will investigate the efficacy of WebGMT,

contrasting pre- versus post-training performance on standard clinical assessments as well as

functional outcome measures of real-world goal attainment.

Materials and Methods

Participants

For this pilot study, we implemented convenience sampling wherein participants were selected

based on their clinical availability. This method is appropriate for exploratory research and for

populations where recruitment is difficult due to low prevalence and/or difficulty with patient

32

access (Stevens, 1996). Our target population for this preliminary study of WebGMT was

persons diagnosed with or at high risk of cerebrovascular disease. The typical pattern of

cognitive dysfunction in this population is comprised of executive control deficits in the context

of relatively preserved memory functioning, making this an ideal cohort for a cognitively

demanding, strategy-based intervention such as GMT. Two WebGMT groups were conducted

each consisting of four participants (N=8; see Table 4.1).

The eight participants were recruited from Sunnybrook Health Sciences Centre and Baycrest

Centre for Geriatric Care in Toronto, Canada, as well as through the local community.

Participants were selected based on the following inclusion criteria: age ranging between 50-80,

diagnosis or physician report of at least one of the following: transient ischemic attack (TIA),

white matter disease (WMD), CVD, stroke risk factors (i.e., hypertension). Participants also had

to have evidence of executive function difficulty as noted in patient records or endorsed during

intake interviews, including two elders with a past history suggestive of attention deficit

hyperactivity disorder (ADHD). All participants were required to have a home computer, high

speed internet service and at least some experience with using a computer and accessing the

internet.

Table 4.1 Participant Characteristics

Subject ID Gender Age Education Recruitment Site Medical Conditions

WB-GMT-01 F 72 18 Sunnybrook TIA/WMD/hypertension

WB-GMT-02 M 70 18 Sunnybrook TIA/WMD/hypertension

WB-GMT-03 M 69 20 Community TIA/hypertension

WB-GMT-04 F 74 13 Sunnybrook TIA/WMD/hypertension

WBCVD2001 M 61 19 Baycrest Hypertension

WBCVD2002 F 77 17 Baycrest TIA/WMD/hypertension

WBCVD2003 M 75 23 Community ADHD/hypertension

WBCVD2004 F 65 16.5 Baycrest ADHD/hypertension

Patients were not excluded based on performance on traditional neuropsychological tests of

frontal lobe function, as research shows that individuals may do well on these standardized

assessments and still exhibit executive function impairments in everyday life (Burgess, et al.,

2006; Chan, Shum, Toulopoulou, & Chen, 2008; Stuss & Levine, 2002). Exclusion criteria

33

included: inability to fluently communicate in English, primary memory deficit and any other

cognitive or functional impairment limiting ability to participate in the training sessions.

All participants reported having hypertension, which is a known risk factor for CVD. All

participants reported functional impairments such as problems with attention, word finding,

absentmindedness, problems with organization, and problems concentrating. Five of the total

number of participants had a stroke or displayed evidence of white matter disease on brain scans

obtained from medical records. The three remaining participants had self-reported executive

control dysfunction and two of these had been diagnosed ADHD, in which executive control

deficits are a hallmark (Proal, et al., 2011).

The study was approved by the Baycrest Research Ethics Board as well as the Sunnybrook

Research Ethics Board. All participants provided written informed consent prior to participation.

Goal Management Training

All participants participated in ten two-hour sessions of GMT (see table below for highlights of

each training module) over five weeks. GMT was delivered using both didactic methods,

involving scripted sessions, as well as more experiential learning opportunities, including group

discussions and in-class exercises following a GMT manual (Levine, Manly, & Robertson,

2008). The primary objective of the intervention was to equip participants with a step by step

strategy that could readily be adopted in their everyday lives. All participants were trained to

‘stop-state-split’ wherein they recognized off-task behaviours, stated their relevant goals and

conducted a means-end analysis to ensure progress towards the goal is maintained (Levine et al.,

2008). In addition to in-class sessions, they were required to complete approximately 20 hours of

at-home assignments, where they were taught to identify and incorporate learned strategies into

their everyday life.

Web Platform

The Adobe Connect videoconferencing platform for meetings and e-learning was the web-

platform selected for the WebGMT protocol. It is an on-line, real-time meeting platform

consisting of features such as screen sharing, high-quality video conferencing, recording

capabilities, notepad sharing, and group chat options. All users received private accounts on this

platform and accessed scheduled meetings through a host web-site created by Dr. Marziali’s

34

research team. The host web-site served as a simple gateway allowing the facilitator to schedule

and post meeting links for participants. In addition, all ten sessions were recorded and saved on a

password secured server for further qualitative analyses.

Procedures

Upon participant recruitment, and prior to start of study, the GMT facilitator met with each GMT

group member in person at the Baycrest site. The objective of this meeting was to a) provide all

participants with a face to face meeting with the facilitator and one another before virtual

sessions, and b) debrief participants and gauge their comfort with technology through open

discussion. Once all participants had met each other and were informed of future steps, they were

provided with GMT packages consisting of intervention specific materials as well as a web-

camera with built-in microphones. The next step was to create accounts and train both therapist

and participants in accessing and using the videoconferencing software. Training was provided

by an experienced research assistant (RA) from Dr. Marziali’s team, who had worked with an

elderly population in the Caring for Others Internet Group intervention (Marziali & Donahue,

2006). The individual meetings between the RA and participants consisted of phone calls and

one-on-one training for web-camera, microphone and headphone installations the procedure for

logging onto the Adobe Connect software via the Caring for Me host website with the help of a

user’s manual. The user manuals were developed by Dr. Marziali and were used in this study as

a technical handbook to guide participants and as a reference during the training sessions. The

initial training meetings (two for each participant) were completed and additional on-line training

sessions were booked based on participant request and comfort with technology. The therapist

received similar training and was provided with an administrator’s manual to host on-line

meetings. Once the GMT facilitator was confident and comfortable with the technology, the

WebGMT group sessions were scheduled.

For participants, attending the GMT sessions via the described web-based platform entailed

logging into Adobe Connect with the web-camera and microphone and “tuning-in” to the shared

screen, where the facilitator shared the GMT modules referencing the provided workbook and

package materials whenever necessary. The participants were able to answer questions via voice

or chat options and to follow notes provided by therapist using the notepad option. The three

sessions were also attended by the RA in order to address any technology related problems as

35

they occurred. Once several sessions were completed with the RA present, the participants were

able to troubleshoot common problems such as a brief disconnection or inability to hear other

participants on their own, and the RA’s input was no longer necessary. In addition, for some

interactive tasks, the GMT instructions were altered to accommodate the delivery mode (for

example, if the task required that participants work in pairs, the instructions were tweaked such

that the task was performed by participants individually). The homework assignments were also

adjusted from requiring participants to remember to come to class with their workbooks to

remembering to send the facilitator an email one day before scheduled session. It is also worth

noting that the concepts behind the interactive tasks and homework assignments were not altered

(e.g. absentmindedness was demonstrated in both tasks whether with a partner or individually).

For qualitative comparison purposes (objective 1), the WebGMT groups were rated by two

independent raters using a devised outcome tool assessing for competence. The first rater was an

occupational therapist who was trained in the GMT protocol and she rated both WebGMT

groups. The second rater was different across both groups, for the first WebGMT group, the rater

was a Master’s student and for the second group, the rater was a PhD student. Both second raters

were in the cognitive rehabilitation field of study and were familiar with the GMT intervention.

In addition, to assess impact of delivery mode on effectiveness of cognitive rehabilitation, a

within-subject comparison method was carried out using neuropsychological tests, self-reported

measures and functional outcome measures.

The WebGMT approach was administered in two separate trials on groups of four participants.

Both groups received identical GMT administration, but the battery of tests administered pre-

and post-intervention was slightly different. Specifically, WebGMT group 1 did not perform the

Tests of Everyday Attention as it was not initially incorporated within the study protocol. As the

objective was to focus on feasibility of administration in the first trial, the quantitative outcome

measures were not emphasized; the participants in the first group did not return the self-report

questionnaires and opted out of an optional GAS follow-up. Running the first WebGMT group

provided insights on logistics for the second WebGMT group as well as experience for the

therapist in using the technology.

Data Analysis

Feasibility

36

All web-based GMT sessions were archived on a secured server where they were accessible via

username and password. Web-based recordings were analyzed using the devised adherence-to-

protocol scale for GMT adapted from the Cognitive Therapy Scale-Revised (CTS-R), a measure

of therapist competence in Cognitive Therapy (CT). The CTS-R contains 12 items covering

various features of therapy; some are general items such as adherence to set agenda, feedback,

collaboration, pacing and efficient use of time as well as interpersonal effectiveness, while others

are CT specific features that were irrelevant to GMT such as eliciting cognitions, guided

discovery and eliciting appropriate emotional expression (Blackburn, et al., 2001). The general

items were unchanged as they could apply to any therapy with a pre-set protocol. The CT

specific items were taken out and replaced with items important to GMT and derived from the

intervention. For instance, a participant engagement item was added to capture group interaction.

All nine items on the newly devised scale, The Cognitive Rehabilitation Therapy Scale for Goal

Management Training (CRTS-GMT), were rated along a six-level model of competence adapted

from the Dreyfus level of competence model (Dreyfus, 1989), across a seven-point Likert scale

ranging from 0- where the therapist did not adhere to that aspect of therapy, to 6- where there is

adherence at very high skill (Likert, 1932). The two raters described above, with good

knowledge of GMT, were selected to watch and rate randomly selected sample sessions. As

described earlier, two WebGMT groups were run. Three sample sessions of the first, and five

sample sessions of the second group were rated by two different raters. In order to establish

competence, a minimum score of 27 on the CRTS-GMT was set as the criterion; this averages to

a score of three on each item (Blackburn, et al., 2001). Data was analyzed by establishing inter-

rater reliability and calculating percent agreement. The objective was to ensure that critical

therapeutic aspects of the GMT intervention were addressed in a similar manner in the online

environment as they were in the classroom.

Efficacy

The efficacy of WebGMT was assessed via a battery of standardized and experimental

neuropsychological tests of executive functioning administered immediately before the

intervention and immediately after to detect post-training cognitive changes. Standardized

laboratory tests may not pick up on cognitive changes manifested in the everyday lives of

individuals, hence functional outcome measures are crucial in determining various changes that

37

are otherwise undetected (Dawson, Anderson, Burgess, Cooper, Krpan, & Stuss, 2009). The

battery comprised of these functional outcome measures; a self-report questionnaire to assess

changes in daily life as well as the Goal Attainment Scale (GAS) measure. All assessments were

administered by the graduate student, who was also a trained psychometrist.

The neuropsychological battery comprised the standardized tests mentioned previously such as

trail making A and B to assess visual attention and task switching, digit span for attention and

verbal working memory (Wechsler, 1985), auditory consonant trigrams for verbal working

memory and divided attention (Brown, 1958) as well as self-ordered pointing task for planning

and working memory (West, Ergis, Winocur, & Saint-Cyr, 1998). To assess frontal cognitive

processes such as planning, problem solving, learning and maintenance of instructional sets, two

versions of the tower task were used in this study to eliminate practice effects by alternating

items from the original task (Delis, Kaplan, & Kramer, 2001; Levine, et al., 2011). Further tests

from the Delis-Kaplan Executive Function System were used: tests to assess verbal fluency,

examining frontal and temporal regions; design fluency tests to assess fluency in spatial domain;

and an additional measure of attention, inhibition and switching using the Stroop test (Delis,

Kaplan, & Kramer, 2001).

Self-reported changes were determined using the Dysexecutive Questionnaire (DEX), a 20-item

questionnaire designed to assess changes in motivation, personality, cognition and behavioural

changes (Burgess, Alderman, Evans, & Emslie, 1996). To measure clinically important change

in cognitive rehabilitation, we administered the GAS which is a patient-centered assessment tool

that enables the quantification of progress towards attainment of goals relevant and meaningful

in participants’ everyday lives (Rockwood, Joyce, & Stolee, 1997). It is a standardized method

which has proven more robust than various other functional outcome measures of cognitive

rehabilitation (Rockwood, Joyce, & Stolee, 1997).

Results

Feasibility- Qualitative Measures

CRTS-GMT. The two sets of ratings for group 1, sessions 2, 5 and 9, and for group 2, sessions

1, 3, 5, 6, and 9 exceeded the minimum score of 27, indicating competence in WebGMT delivery

38

(see Table 4.2 and Table 4.3). The average total across all sessions for both raters was 47 (SD=

2.8) for group 1 and 48 (SD= 3.3) for group 2. Inter-rater reliability was calculated using the

intra-class correlation statistic for quantitative measurements and yielded an ICC of 0.49,

indicating a fair agreement, between raters of group 1, and an ICC of 0.59, indicating a moderate

agreement between raters of Group 2 recordings. Percent agreement between raters was

calculated to be 56% for WebGMT group 1 and 64% for WebGMT group 2 with an overlap of

plus and minus 0.5 points across the ratings.

Table 4.2 WebGMT Group 1 CRTS-GMT scores of two raters across randomly selected session

recordings

Table 4.3 WebGMT Group 2 CRTS-GMT scores of two raters across randomly selected session

recordings

Session 1 3 5 6 9

Rater 1 2 1 2 1 2 1 2 1 2

1. Adherence to module outline 3.5 6 3.5 6 6 6 6 6 6 6

2. Feedback 6 5 6 6 6 6 6 6 5.5 5

3. Collaboration (verbal and non-verbal) 5.5 4 5.5 5 6 6 4.5 5 5 5

4. Pacing and efficient use of time 3.5 5 3.5 3 6 3 6 4 6 5

5. Interpersonal effectiveness 6 6 6 6 6 6 6 6 6 6

6. Conceptual integration 5.5 6 6 5 6 6 6 6 6 6

Session 2 5 9

Rater 1 2 1 2 1 2

1. Adherence to module outline 6 5 4 5 4.5 4

2. Feedback 5.5 5 6 6 6 6

3. Collaboration (verbal and non-verbal) 5.5 2 3 2 5 5

4. Pacing and efficient use of time 5 4 5 5 5 4

5. Interpersonal effectiveness 5.5 5 6 6 6 6

6. Conceptual integration 5.5 4 6 6 6 6

7. Therapist Skills 5 6 5 6 6 6

8. Homework Setting 5 6 5 6 6 5

9. Participant Engagement 4.5 5 5 5 5.5 6

Total (max. 54) 47.5 42 45 47 50 48

Percent Agreement between raters 56%

Intraclass correlation 0.49 (fair agreement)

39

7. Therapist Skills 5.5 6 6 6 6 5 6 6 6 5

8. Homework Setting 5 6 3 6 6 6 6 6 ** 6

9. Participant Engagement 5 4 5 4 6 5 5.5 6 5 5

Total (max. 54) 45.5 45.5 44.5 44.5 54 49 51 51 45.5 49

Percent agreement between raters 64%

Intraclass correlation 0.59 (moderate agreement)

Feedback Survey. The feedback survey showed that seven out of the eight participants had

very or somewhat positive feedback on each item of the survey. In addition, five of the total

number reported that they would prefer a WebGMT group over a traditional in-class GMT

intervention (see Table 4.4).

Table 4.4 Participant Feedback Survey

Questions V S N NV NA

Overall how easy or difficult did you find the use of Adobe

Connect?

XXX XXX

X

X

Overall how helpful was the technical support offered? XXXXX

X

X X

How helpful did you find Adobe Connect specifically in the

learning experience?

XXXXX XX X

If you were to participate in this brain training program, would you rather sign up to a web-based or

in-class group?

5 x Web-based

3 x In-class

NOTE: V=very, S=somewhat, N=neutral, NV=not very, NA= not at all (NA for Question 2 =Not

applicable)

Efficacy- Pre-Post Intervention Measures

Neuropsychological Tests of Executive Functions. Reliable changes indices (RCI) were

calculated for the Delis-Kaplan Executive Function System (DKEFS) measures (see Table 4.5)

as these had published test-retest reliability information. As can be seen in Table 6, few

individual measures showed significant RCI values from pre-to post-training. Similarly, no

significant results (p > .05 all tests) were observed on t-tests of for pre- versus post-GMT

neuropsychological data (Table 4.6).

40

Table 4.5 DKEFS raw score significance using the reliable change index

Reliable Change Index 1 2 3 4 5 6 7 8 Mean 1 (SD) Mean 2 (SD)

Phonemic Fluency NS NS NS NS NS NS NS NS 40.8 (18.0) 45.4 (16.6)

Semantic Fluency NS NS NS NS NS NS NS NS 41.4 (9.59) 38.8 (9.3)

Design Fluency 1 NS S NS NS NS NS NS S 10.0 (2.3) 9.8 (4.2)

Design Fluency 2 NS NS NS NS NS NS NS NS 11.3 (2.4) 11.3 (2.3)

Design Fluency 3 NS NS NS S NS NS NS NS 7.8 (4.3) 8.4 (1.8)

Stroop NS NS NS NS NS NS NS NS 77.6 (23.9) 73.1 (25.8)

Stroop Switching NS NS NS NS NS NS NS NS 86.1 (29.0) 80.4 (18.4)

Table 4.6 Neuropsychological tests of executive function raw score significance using Student’s

t-test

Executive Function

Domain

Assessment N t-test

significance

Mean 1 (SD) Mean 2 (SD)

Working memory

SOPT 8 NS 10.6 (4.1) 11.6 (7.5)

Consonant

Trigrams

9 seconds 4 NS 9.3 (5.1) 8.8 (3.4)

18 seconds 4 NS 9.8 (5.5) 5.3 (3.8)

Processing Speed

Trails A 8 NS 33.6 (8.4) 36.2 (8.4)

Task Switching Trails B 8 NS 114.9 (47.8) 111.7 (47.3)

Attention Digit Span 8 NS 17.1 (4.1) 16.8 (4.3)

Tests of Everyday Attention 4 NS 6.3 (1.9) 5.5 (1.3)

Planning Tower Task- Achievement

score

8 NS 9.8 (1.8) 9.9 (2.3)

Questionnaires. There were no significant effects from within-subject t-test analyses (P > .05,

all tests) on the five DEX scales or overall scores. Table 4.7 shows group means across self and

other scores of specific DEX scales.

Table 4.7 Dysexecutive Questionnaire raw scores using Student’s t-test

Questionnaire

Component

Inhibition Mean 1 (SD) Mean 2 (SD) Intention Mean 1

(SD)

Mean 2

(SD)

Self t-test NS 11.0 (4.4) 12.333 (5.0) NS 7.7 (3.2) 8.3 (1.2)

Other t-test NS 8.0 (7.0) 7.000 (6.9) NS 6.0 (4.6) 4.0 (3.6)

Questionnaire

Component

Executive

Memory

Mean 1 (SD) Mean 2 (SD) Positive

Affect

Mean 1

(SD)

Mean 2

(SD)

Self t-test NS 2.0 (2.6) 3.000 (3.0) NS 5.0 (1.0) 5.333 (2.1)

41

Other t-test NS 1.7 (1.2) 1.333 (0.5) NS 3.7 (2.1) 4.000 (2.0)

Questionnaire

Component

Negative

Affect

Mean 1 (SD) Mean 2 (SD) Overall Mean 1

(SD)

Mean 2

(SD)

Self t-test NS 3.0 (1.0) 3.667 (2.3) NS 28.7 (7.5) 32.7 (9.0)

Other t-test NS 1.3 (0.6) 1.333 (1.5) NS 20.7 (12.1) 21.7 (9.5)

Goal Attainment Scale (GAS). For the four participants in the WebGMT group 2, GAS scores

(see table 4.8) showed that all participants achieved expected level (N=1) or higher (N=3) on at

least one of their established goals.

Table 4.8 Goal Attainment Scale measure

Subject Summary of Goal Score Achievement Level

WBCVD2001 Goal 1

Reduce incidence of word-finding difficulty in conversations by 40%-

50%. Be able to find the right word in conversations about half the time

0 Expected level

WBCVD2002 Goal 1

Use the same book to track tasks/lists at least 50% of the time

2 Much better than

expected level

Goal 2

Complete at least half of required tasks without being distracted or

starting a new task

1 Somewhat better than

expected

WBCVD2003 Goal 1

Begin at least 50% of all projects on time

Complete at least 40% of all projects on time

1 Somewhat better than

expected

WBCVD2004 Goal 1

Get into bed by midnight at least 4 out of 7 nights in the week

-1 Somewhat less than

expected

Goal 2

Prepare lunch/dinner in advance and take it along at least 3 out of 5

work-days in the week

2 Much better than

expected

Discussion

This pilot study is, to our knowledge, the first investigation of remote, web-based and real-time

delivery of a small group cognitive training intervention (GMT). There were two primary

research objectives: i) to evaluate the feasibility of this innovative delivery approach and ii) to

assess the efficacy of WebGMT for remediating executive control deficits.

42

Objective 1: Feasibility

The semi-quantitative-qualitative adherence-to-protocol rating scale adapted from the CTS-R

(Blackburn, et al., 2001) demonstrated that GMT can be delivered effectively via a web-interface

without compromising learning objectives while maintaining a high level of therapeutic integrity

(therapist competence, participant engagement and adherence to protocol). The analyses of

sample sessions demonstrated that across two raters the CRTS-GMT totals far exceeded the

minimum level of competence set at the midpoint level for the rating scale. The inter-rater

reliability across all sessions from both web-based intervention groups demonstrated fair to

moderate agreement, deeming the results reliable. Percent agreement on specific items between

raters ranged from 55%-65% for both groups. When total scores were closely examined, the

difference between the totals of two raters was minimal (0-10, with a total of 52); therefore,

although raters do not agree exactly on the score of specific items, the disagreement is not drastic

enough to make large changes in total scores. Moreover, as a supplementary analysis (see

appendix C), we contrasted these therapist ratings with ratings of previously recorded in-class

GMT sessions from an earlier study of Multiple Sclerosis patients. While this was purely a

speculative analysis, given the non-matched patient samples and different therapists, it is

nonetheless encouraging that a large majority (72%) of ratings were equal to or higher for the

WebGMT delivery versus the traditional in-class format. These preliminary data provide strong

support for WebGMT as a feasible strategy for increasing access to GMT and potentially other

cognitive rehabilitation programs where time, cost, geographic or physical barriers may impede

participation.

To further support the feasibility of this approach, surveys completed by research participants

contained positive feedback, suggesting that this novel approach did not impede participation in

a much needed cognitive rehabilitation intervention. When subjects were asked whether they

would choose a web-based intervention versus a traditional in-class intervention, five out of the

eight participants chose the web-based platform. This level of positive affirmation from subjects

who had no previous experience with this type of web-interface supports the notion that the

remote delivery approach was not a significant disincentive for subject participation; indeed, for

over half of the participants, the loss of in-person contact was more than offset by the reduction

in travel time afforded by WebGMT.

43

Previous research examining the feasibility of web-based platforms in psychosocial support

groups addressed this objective through extensive qualitative methods (Marziali & Donahue,

2006; Marziali, 2006; Marziali, Damianakis, & Donahue, 2006). This study followed a different

approach in devising a brief 9-item scale assessing therapist competence of following the GMT

outline and participants’ level of engagement with the therapist and as a group. As a future

direction, this tool can also be used in training therapists to administer GMT, whether through a

web-based platform or in traditional in-class methods, ensuring GMT is delivered consistently

irrespective of location, therapists or delivery modality.

Objective 2- Efficacy

We used a similar battery of neuropsychological tests and functional outcome measures as had

been reported in previous GMT studies (Levine, et al., 2007; van Hooren, et al., 2007;

Schweizer, et al., 2008; Levine, et al., 2011) to assess executive functioning pre- and post-

WebGMT. Results were consistent with these previous studies wherein few significant changes

were identified on standardized neuropsychological measures. This is likely due to the small

sample size of this pilot study, high baseline performance in our samples, and insensitivity of

standardized executive function measures (Levine & Downey-Lamb, 2002).

The insignificant DEX findings observed here reflect those reported in the most recent GMT

study (Levine, et al., 2011). As reported by Levine and colleagues, we suggest that the increase

in symptoms (albeit insignificant) observed here may reflect increased insight or awareness of

cognitive impairments by the participant and their significant other causing them to increase item

endorsement.

Here we also used the GAS, a functional outcome measure of goal attainment in the daily life of

participants. This measure, which has not previously been reported in conjunction with GMT

studies, is a semi-quantitative tool that enables researchers to obtain a standardized measure of

goal attainment across a wide range of goals and individuals. It is a standardized method of

quantifying achievement and has proven to be a sensitive measure of cognitive rehabilitation

efficacy (Rockwood, Joyce, & Stolee, 1997). Given that the emphasis of GMT is on re-

establishing goal-directed behaviour for persons with executive dysfunction, the positive results

obtained here are particularly promising for WebGMT as an efficacious training protocol that

can effect change in the lives of participants.

44

Conclusion

This pilot study is the first of its kind to present the administration and delivery of group-based

therapy using current advances in technology. Accordingly, this study examines the feasibility

and effectiveness of WebGMT, an approach to delivering GMT using web-based real-time

videoconferencing software that has not been previously undertaken. To the best of our

knowledge, this is also the first study to employ an adherence-to-protocol scale, a newly

developed semi-quantitative-qualitative tool devised to analyze various aspects of therapy. We

administered WebGMT to two groups of participants with CVD or with CVD risk factors.

Preliminary data from the devised semi-quantitative-qualitative scale (CRTS-GMT) showed that

this approach is feasible and that factors such as group engagement and adherence to GMT

protocol are not compromised. In addition, the efficacy of this approach was analyzed through a

battery of neuropsychological and functional outcome measures. We found that, consistent with

previous research, participants showed significant improvement through functional outcome

measures and no significant changes through neuropsychological measures. These results in

combination can be used as the stepping stone for further research investigating this innovative

delivery mode. This data can also provide us with guidance to help determine effect sizes for

future randomized controlled trials. Moreover, future studies can further develop the CRTS-

GMT tool to use as a training tool for therapists administering GMT in traditional and web-based

modes.

45

Chapter 5 Discussion

The main purpose of this study was to investigate the feasibility and efficacy of delivering group

cognitive rehabilitation therapy using a web-based platform for people with executive function

impairments associated to cerebrovascular risk factors and/or cerebral small vessel disease. This

study was important to undertake in order to introduce a novel approach to improve the

accessibility of cognitive rehabilitation interventions. As mentioned previously in Chapter 2,

Goal Management Training (GMT) is an evidence-based cognitive rehabilitation intervention

(Levine, et al., 2000; Levine, et al., 2007; Fish, et al., 2007; Levine, et al., 2011) that requires

participants to attend ten 2-hour sessions over five weeks, placing time and travel burdens on

patients that may limit accessibility and decrease patient engagement and participation. To

address these access barriers, this study investigated the feasibility and efficacy of web-based

administration of GMT (WebGMT). Cerebrovascular Disease (CVD) patients were selected as

the target population for this pilot intervention study. Executive dysfunction in the presence of

relatively preserved memory functioning is a common cognitive profile of CVD, and these

patients are well placed to benefit from a strategy-based, executive training program.

Although increasingly common in individualized cognitive behavioural therapy and in

psychosocial support groups, to the best of our knowledge, no one has examined the delivery of a

group-based cognitive intervention in real-time using a web-based platform (Bouchard, et al.,

2004; Marziali, 2006; Marziali & Donahue, 2006). In this study, two WebGMT sessions were

implemented to evaluate the feasibility and efficacy of delivering GMT. Feasibility was

primarily assessed using a semi-quantitative-qualitative scale of adherence to protocol,

developed from a cognitive behavioural therapist rating instrument (Blackburn, et al., 2001).

Various therapeutic aspects of the GMT training protocol, including group engagement,

therapist-participant interactions, etc. were assessed (See Appendix A). Furthermore, I created a

self-report feedback survey through which participants were able to report the usefulness of

technology as well as their overall preference for intervention delivery (in-class versus web-

based - see Appendix B). Based on the results of these preliminary feasibility analyses, I

concluded that this innovative virtual delivery approach for delivering GMT could be

implemented successfully in a patient population, closely adhering to the established therapeutic

protocol and without compromising participant engagement.

46

To address the efficacy question, I adopted a within-subject, longitudinal design in which I

administered a battery of neuropsychological and functional outcome measures at baseline (pre-

intervention) and post-intervention, using cognitive assessment protocols adopted for previous

GMT studies with acquired brain injury and normal aging populations (Levine, et al., 2011; van

Hooren, et al., 2007). No significant changes were observed on standardized neuropsychological

measures, as has been reported in previous GMT studies, but in this preliminary proof-of-

concept study, likely reflecting the small sample size and limited statistical power to detect

changes. Nevertheless, preliminary results from a real world measure of goal attainment (Goal

Attainment Scale- GAS, see Chapter 3), indicated that patients participating in WebGMT were

able to achieve real world goals at a level that exceeded expectation – a critical measure of GMT

success.

This final chapter provides a summary of my thesis project, bringing together the literature

review in Chapter 2 and the findings from the study. The goal of this summary is to provide the

reader with conclusions from this proof-of-concept intervention project as well as lay out

directions for future research. First, I briefly review the relevant background information on

executive control functions and cognitive rehabilitation for the population at hand. Specifically, I

relate the findings from the efficacy study presented here to the results of previous research with

GMT. Second, I provide an overview of existing research using similar web-delivery platforms

in a therapeutic context, highlighting the novelty and promise of this delivery platform for GMT

and other cognitive therapy interventions based on the results obtained in this pilot project.

Finally, I address limitations of the research and provide recommendations for future studies.

5.1 Executive Function

My thesis project focused on GMT, one of the few cognitive rehabilitation interventions with

demonstrated efficacy for remediating executive function impairments associated with normal

aging, as well as acquired brain injury. In reviewing the results of the study, it is important to

have a clear understanding of the cognitive domain involved as well as the theories that the

intervention is based upon. In Chapter 2, I defined executive function as a set of higher cognitive

processes that allow an individual to problem solve, maintain attention, demonstrate behavioural

control, plan and monitor on-going actions as well as adapt to novel situations (Levine, Turner,

& Stuss, 2008). These skills, amongst others, are dependent on the integrity of the frontal lobes

47

and frontally mediated systems and, as such, are highly susceptible to focal brain changes that

affect this area directly (e.g. stroke) or distributed neuropathological changes impairing multiple

brain regions or the connections between them (e.g. small vessel disease) (Stuss & Alexander,

2000). Not surprisingly, diffuse ischemic white matter changes secondary to small vessel disease

are often associated with executive dysfunction, including reduced goal-directed behaviours

(Verghese, Wang, Katz, Sanders, & Lipton, 2009; Black, 2007). While many schemas have been

proposed to characterize executive control functions, in this study I have characterized executive

control specifically in terms of goal-directed cognitive processing. This characterization of

executive control emerges from three theoretical models of executive function that converge

around an understanding of executive dysfunction as a reduction in goal-directed cognition

leading to impairment of a patient’s ability to maintain focus and avoid off-task behaviours. This

theoretical framework serves as a foundation of the current study, wherein the intervention

protocol explicitly targets goal-directed behaviour in a patient population with known diffuse

brain changes (small vessel disease), and concomitant executive dysfunction.

The three distinct models of executive function reviewed in Chapter 2 and those that inform the

development of GMT are: the supervisory attentional system (SAS) by Shallice and Norman

(1980), Baddeley's model of working memory (1986), and Duncan's model of goal neglect

(1986). Shallice and Norman (1980) define executive functioning as an attentional system which

mediates the activation of existing schemas or the formation of new schemas depending on

whether the cognitive context is routine or novel, with the supervisory attention system (SAS,

i.e., executive control) engaged by the latter but not the former. Baddeley's theory of working

memory expanded on this model of the SAS by introducing the concept of a central executive

system (CES), which is necessary for flexible goal maintenance in the context of distractions that

may promote maladaptive or off-task behaviours. These models were seminal in advancing our

understanding of executive control functioning as an attentional control system, necessary for

maintaining goal-directed behaviours in non-routine, novel and distracting environments.

However, a third model proposed by Duncan, based on studies of patients with frontal lobe

injury, more directly conceptualized executive control functions in terms of goal-directed

behaviour. This theory characterized executive dysfunction as a separation of knowledge from

intended action, resulting in a failure of goal selection and maintenance ultimately leading to

goal neglect (Duncan, 1986). This work directly informed the development of GMT, a cognitive

48

rehabilitation protocol explicitly targeting goal neglect and failures to maintain goal direction

following frontal system dysfunction.

The review of the cognitive intervention used provides a clearer explanation of why the models

reviewed above are important to the study undertaken. In addition, the definition of executive

function helps the reader to understand specific outcome measures used and their relevance in

measuring cognitive domain.

5.2 Efficacy of Web-Based Goal Management Training (WebGMT)

The cognitive rehabilitation intervention used in this study is GMT. GMT was originally

developed by Robertson (1996), and has been further elaborated as a multi-session, small group,

cognitive therapy protocol by Levine, Manly and Robertson (2008). GMT is an executive control

function training protocol that emphasizes the re-engagement of endogenous attentional control

processes. It promotes the formulation of explicit goal lists and the use of means-ends analysis to

ensure behaviour is constantly guided by its intended outcome (i.e., the goal). Since the earliest

formulation of this intervention approach, many studies have been conducted on various patient

populations such as acquired brain injury (stroke and TBI) (Levine, et al., 2011; Levine, et al.,

2000; Fish, et al., 2007), focal cerebellar damage (Schweizer, et al., 2008) and normal aging

populations with higher cognitive complaints (Levine, et al., 2007; van Hooren, et al., 2007).

These studies have provided convincing evidence that GMT is effective for remediating

executive control. Indeed. in a recent review of cognitive rehabilitation interventions, GMT was

one of the only executive function intervention strategies recommended as a practice standard

(Cicerone, et al., 2011). All of these studies utilized similar forms of the GMT protocol, adopting

a strategy-based approach to remediate executive control dysfunction and increase goal-directed

behaviour in everyday life. While the efficacy of GMT has been amply demonstrated in a

research context, translation of the intervention protocol from the laboratory to clinical settings

remains a considerable challenge owing to the significant time, travel and potential economic

burdens associated with participation.

My project investigated the feasibility of reducing these barriers by delivering GMT remotely via

a web-based platform. One of the objectives of this study was to use neuropsychological and

functional outcome measures in order to demonstrate the efficacy of this novel WebGMT

49

approach. To address this question, I used a combination of standardized and experimental

neuropsychological measures, the results of which failed to show statistical significant changes

after pre-post intervention. While the small number of subjects recruited for this pilot study

limited the statistical power for the group analysis, the failure to observe significant changes in

standardized neuropsychological measures is generally consistent with previous research on

GMT. For example, in a normal aging population the Stroop test, a measure of inhibition, did not

show any significant results after GMT and only a few significant improvements were shown on

other standardized measures (van Hooren, et al., 2007). In a recent study of GMT in a healthy

aging cohort, van Hooren and colleagues (2007) attributed their null findings on

neuropsychological testing to the lack of significant impairment on the measures prior to

training, thus limiting the capacity for improvement. The same conclusion can be made for the

results observed in this study. The participants in both intervention sessions displayed high levels

of cognitive performance at baseline and therefore significant changes after intervention were

less likely. Furthermore, standardized measures of frontal lobe function are notoriously

insensitive to executive function deficits in real world settings (Shallice & Burgess, 1991) and

thus would not be expected to show training-related changes (Levine, et al., 2011).

For this reason, the majority of studies investigating the efficacy of executive function

interventions include functional outcome or ‘real-world’ measures, including simulated real life

tasks (Levine, et al., 2000), complex tasks (e.g. Multiple Errands Test) (Dawson, Anderson,

Burgess, Cooper, Krpan, & Stuss, 2009), and self-reported questionnaires (e.g. the dysexecutive

questionnaire) (Burgess, Alderman, Evans, & Emslie, 1996). All of these functional outcome

measures showed improvements following GMT (Levine, et al., 2011; Fish, et al., 2007;

Schweizer, et al., 2008). Given the sensitivity of these measures to GMT intervention, I included

two functional outcome measures, the DEX as well as a novel functional assessment not

heretofore used in GMT intervention research: the Goal Attainment Scale (Rockwood, Joyce, &

Stolee, 1997). The DEX showed no immediate improvement after intervention. This is consistent

with the recent report from Schweizer and colleagues (2008), who suggested that treatment

effects require time for consolidation into the lives of individuals. As this was a pilot

intervention, no long-term follow-up assessments were conducted. Hence, it is unclear whether

this explanation might apply in the current study. In contrast to the DEX outcomes, participants

receiving the GAS reported beyond-expected levels of goal attainment post-GMT.

50

In summary, this study adapted a proven intervention approach for remediating executive control

dysfunction. The overall goal of the research is to reduce barriers to access imposed by the

standard multi-session in-class delivery protocol through the development of a web-based remote

delivery protocol that (i) is feasible to deliver to individuals with executive dysfunction and (ii)

does not compromise therapeutic efficacy. The results of this pilot study provide encouraging, if

preliminary, support for this approach.

5.2.1 Limitations and Future Directions

While the preliminary data presented here supports the feasibility and efficacy of WebGMT,

future research will need to address several limitations of the current study to confirm the results

presented. First, the participants involved in both WebGMT sessions were a convenience sample

and did not demonstrate clinical impairments on standardized assessment. Therefore, as

mentioned previously, room for improvements on tests of executive functions was limited. In

order to demonstrate improvement in the future, a larger, prospective cohort with greater

variability in baseline performance as well as variability in level of education should be

recruited. Given that this study was a pilot study focusing on feasibility, the lack of a randomized

control trial (RCT) was acceptable; the foundation has successfully been laid for future studies in

which an RCT is used to directly contrast WebGMT with traditional in-class delivery methods.

In addition, the ultimate goal of an intervention is to demonstrate changes in daily behaviour;

that is, functional outcome measures should be focused on, in order to capture changes that are

otherwise undetected using traditional neuropsychological assessments of higher cognitive skills.

Furthermore, in a CBT study, Arundine and her colleagues (2012) observed effects of the

therapy 6-months after treatment, providing evidence that therapy impact may be observed after

a certain time duration which may apply to GMT studies. While preliminary functional outcome

data was presented here, future studies will need to collect multiple functional outcomes with

longer-term follow-up assessments to measure the durability of the training outcomes.

5.3 The importance of WebGMT

In Chapter 2, I highlighted the obstacles that may impede participation in GMT. I also presented

the International Classification of Functioning (ICF), providing a framework to further explain

various factors interacting to affect one’s daily life (Vrankrijker, 2003). In this study, the health

condition is executive dysfunction primarily due to vascular risk factors and small vessel disease

51

including two elders with ADHD in addition to vascular risk factors. This impairment interacts

with body function and activity to hinder the individual from leading a normal lifestyle in which

they are able to plan and carry out activities as successful as they want or need to. Other

contextual factors, such as distance from urban centers as well as limited access to transportation

can hinder access to treatment that may improve the health condition and in turn increase

participation in society. In the standard cognitive intervention that is the basis of the one

implemented here, such barriers are present. GMT poses time and travel burdens on participants,

making it difficult for patients in need to access and fully engage in the treatment protocol. In

order to overcome these barriers, the WebGMT approach was introduced as a novel way of

delivering cognitive rehabilitation to people otherwise unable to access treatment. This method

required patients to have access to high-speed internet and be comfortable with using the video-

conferencing technology. Two WebGMT groups met online through the Adobe Connect

platform and attended a training program over five weeks of treatment. This new method proved

feasible and initially successful as determined via tools to assess platform delivery. In the

following section, I provide an overview of the qualitative data collected and specific features

that were assessed as fundamental aspects of GMT to gauge the feasibility of delivering

WebGMT.

5.4 Feasibility Web-Based Goal Management Training

One of the objectives of this study was to assess the feasibility of delivering WebGMT, a group-

based cognitive rehabilitation intervention using a web-based platform. Although online real-

time caregiver support groups have been organized (Marziali, 2006), as well as one-on-one

videoconference cognitive behavioural therapy (Bouchard, et al., 2004), to date no group-based

cognitive therapeutic intervention has been delivered using the web. This study is the first to

implement WebGMT using the Adobe Connect videoconferencing platform, thereby allowing

participants to access this rehabilitation intervention without the accessibility barriers described

earlier. To assess feasibility, two semi-quantitative-qualitative tools were developed: the

adherence-to-protocol scale for GMT and participant feedback surveys.

In Chapter 3, I provided an in-depth explanation of the adherence-to-protocol scale and the

changes that were made in order to transform the instrument from a cognitive behavioral therapy

scale to a cognitive rehabilitation therapy scale for GMT (CRTS-GMT). In analyzing data using

52

this tool, it is crucial to highlight each feature examined and the importance it carries in assessing

various aspects of GMT. Important features of this cognitive rehabilitation intervention include

therapist adherence to GMT module outlines and structure, as well as participant feedback and

effective time utilization. Each of these were assessed using the CRTS-GMT across two raters,

who scored higher than the average score of three. In addition, other features such as group

collaboration, therapist skills, participant engagement and homework setting were looked at and

across sample sessions also scored higher than the average score. The threshold total score for

the CRTS-GMT for each session was placed at 27, above which competence was established.

For both raters familiar with the GMT protocol, the total scores exceeded 27, providing sufficient

preliminary data to indicate no significant compromises in the assessed features of the group

intervention.

Taking this analysis a step further, the WebGMT session recordings were also compared to in-

class GMT sessions using a very similar protocol (see Appendix C). The differences between the

items and total CRTS-GMT scale scores across both platforms of delivery did not differ

significantly, further establishing that no crucial aspects of therapy were compromised. The

percent at which WebGMT scores were higher than traditional in-class GMT ranged between

55% and 83% across two sample sessions, demonstrating that the WebGMT excelled at specific

aspects of therapy. This difference may have been due to variability in therapist and populations

of the WebBased groups versus the in-class GMT groups.

The feedback questionnaire, used to establish participant satisfaction with the therapy and

technology, showed positive responses to questions specific to the learning experience using the

Adobe Connect platform. In addition, five of the 8 participants indicated that they would prefer

using a web-based platform in receiving the intervention.

5.4.1 Limitations and Future Directions

This qualitative data shows that the novel web-based approach to the delivery of GMT is

promising and requires future research focusing on qualitative methods of analyzing this

approach. However, it is important to recognize limitations of this study that may help in

directing future projects using similar approaches. For example, though raters were given

specific instructions that the score of six on CRTS-GMT items should be a rare occurrence (i.e.,

only when there is no chance for improvement) many rating were still at ceiling. This Adherence

53

to Protocol instrument, specifically developed for this study, requires further validation to

determine its utility as a tool for assessing the quality and consistency of intervention delivery,

whether across delivery platforms (as described here), training sites or therapists. More detailed

instructions to raters including a rating manual and pre-rating training sessions would improve

the quality and consistency of ratings. As described above, an RCT design, contrasting adherence

ratings across in-class and WebGMT sessions with similar patient populations and therapist

teams, would be necessary to provide a full validation of the consistency and utility of the rating

tool.

5.5 Summary and Conclusions

This research study explored a novel approach for delivering GMT using a real-time

videoconferencing platform. There were two primary study objectives: to investigate the

feasibility of delivering GMT using this web-based approach and to examine the efficacy of the

WebGMT approach vis-à-vis past research examining GMT efficacy. To address these two

research aims, semi-quantitative-qualitative assessment tools were developed and

neuropsychological as well as functional outcome measures were administered following a

within-subject, longitudinal experimental design. Using an adherence-to-protocol tool developed

for this study, the preliminary qualitative data confirmed that WebGMT was feasible in a

population of cerebrovascular risk patients with reported executive control deficits. Moreover,

preliminary results suggest that GMT can be delivered remotely without compromising

rehabilitation outcomes. This preliminary, proof-of-concept, study has established the empirical

foundation and research tools necessary for larger, randomized controlled trials. These

prospective studies are necessary to provide concrete evidence supporting feasibility and efficacy

delivering group-based cognitive rehabilitation therapy remotely using a web-based platform.

54

References

Alexander, M. D., & Stuss, D. T. (2003). Executive disorders. In E. J. Noseworthy, Neurological

Therapeutics, Principles and Practice (Vol. 2, pp. 2808-2809). London: Martin Dunitz.

Alford, B., & Beck, A. T. (1997). The integrative power of cognitive therapy. New York:

Guilford Press.

Arundine, A., Bradbury, C. L., Dupuis, K., Dawson, D. R., Ruttan, L. A., & Green, R. E. (2012).

Cognitive behavior therapy after acquired brain injury: maintenance of therapeutic benefits at 6

months posttreatment. The Journal of head trauma rehabilitation , 27 (2), 104-12.

Baddeley, A. D. (1986). Working Memory. Oxford: Clarendon Press.

Baddeley, A. D., & Hitch, G. (2000). Development of working memory: Should the Pascual-

Leone and Baddeley and Hitch models be merged? ournal of Experimental Child Psychology ,

77, 128-137.

Bergquist, T., Gehl, C., Holzworth, N., & Beaulieu, W. (2008). Internet-based cognitive

rehabilitation in individuals with acquired brain injury: A pilot feasibility study. Brain Injury ,

22(11), 891-897.

Bilbao, A., Kennedy, C., Chatterij, S., Ustun, B., Luis, J., Barquero, V., et al. (2003). The ICF:

Applications of the WHO model of functioning, disability and health to brain injury

rehabilitation. NeuroRehabilitation , 18, 239-250.

Black, S. E. (2007). Therapeutic Issues in Vascular Dementia: Studies, Designs and Approaches.

The Canadian Journal of Neurological Sciences , 34(1), S125-30.

Black, S., Gao, F., & Bilbao, J. (2009). Understanding White Matter Disease Imaging-

Pathological Correlations in Vascular Cognitive Impairment. Stroke , 40(1), S48-S52.

Blackburn, I. M., James , I. A., Milne, D. L., Baker, C., Standart, S., Garland, A., et al. (2001).

The Revised Cognitive Therapy Scale (CTS-R): Psychometric Properties. Behavioural and

Cognitive Therapy , 29, 431-446.

55

Bouchard, S., Paquin, B., Payeur, R., Allard, M., Pivard, V., Fournier, T., et al. (2004).

Delivering Cognitive-Behavior Therapy for Panic Disorder with Agoraphobia in

Videoconference. Telemedicine Journal and e-Health , 10, 13-25.

Brown, J. (1958). Some tests of the decay theory of immediate memory. Quarterly Journal of

Experimental Psychology , 10, 12-21.

Brown, K. G., Rietz, T. A., & Sugrue, B. (2005). The Effects of Videoconferencing, Class Size,

and Learner Characteristics on Training Outcomes. Performance Improvement Quarterly , 18(1),

59-82.

Burgess, P. W., & Robertson, I. H. (2002). Principles of the Rehabilitation of Frontal Lobe

Function. In D. T. Stuss, & R. T. Knight, Principles of Frontal Lobe Function (pp. 557-572).

New York: Oxford University Press.

Burgess, P. W., Alderman, N., Evans, J. J., & Emslie, H. (1996). "The dysexecutive

questionnaire". In Behavioral Assessment of the Dysexecutive Syndrome. Bury St. Edmunds:

Thames Valley Test Company.

Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates, L. M., Dawson, D. R., et al.

(2006). The case for the development and use of "ecologically valid" measures of executive

function in experimental and clinical neuropsychology. Journal of the International

Neuropsychological Society , 12, 194-209.

Chan, R. C., Shum, D., Toulopoulou, T., & Chen, E. Y. (2008). Assessment of executive

function: Review of instruments and identificaiton of critical issues. Archives of Clinical

Neuropsychology , 201-216.

Chen , A. J., Novakovic-Agopian, T., Nycum, T. J., Song, S., Turner, G. R., Hills, N. K., et al.

(2011). Training of goal-directed attention regulation enhances control over neural processing for

individuals with brain injury. Brain , 134(Pt 5), 1541-54.

Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Fraas, M., et al.

(2011). Evidence-based cognitive rehabilitation: updated review of the literature from 2003

through 2008. Archives of physical medicine and rehabilitation , 92 (4), 519-530.

56

Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Fraas, M., et al.

(2011). Evidence-Based Cognitive Rehabilitation: Updated Review of the Literature From 2003

Through 2008. Archives of Psychical Medicine and Rehabilitation , 92, 519-530.

Damianakis, T., Climans, R., & Marziali, E. (2008). Social Workers’ Experiences of Virtual

Psychotherapeutic Caregivers Groups for Alzheimer’s, Parkinson’s, Stroke, Frontotemporal

Dementia, and Traumatic Brain Injury. Social Work with Groups , 31(2), 99-116.

Dawson, D. R., Anderson, N., Burgess, P., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009).

Further development of the Multiple Errands Test: Standardized scoring, reliability and

ecological validity for the Baycrest Version. Archives of Physical Medicine & Rehabilitation , 90

(11), S41-S51.

Delis, D. C., Kaplan, E., & Kramer, J. H. (2001). Delis-Kaplan Executive Functioning System

(D-KEFS).

Dreyfus, H. L. (1989). The Dreyfus model of skull acquisition. In J. Burke, Competency based

education and training. London: Falmer Press.

Duncan, J. (1986). Disorganisation of behaviour after frontal lobe damage. Cognitive

Neuropsychology , 3(3), 271-290.

Duncan, J. (1986). Disorganization of behaviour after frontal lobe damage. Cognitive

Neuropsychology , 2, 271-290.

Duncan, J., & Stuss, D. T. (2000). Rehabilitation of executive functioning: an experimental-

clinical validation of goal management training. Journal of the International Neuropsychological

Society , 6(3), 299-312.

Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal

lobe: the organization of goal-directed behavior. Cognitive Psychology , 30(3), 257-303.

Duncan, J., Parr, A., Woolgar, A., Thompson, R., Bright, P., Bishop, S., et al. (2008). Goal

neglect and Spearman's g: competing parts of a complex task. Journal of Experimental

Psychology , 137(1), 131-48.

57

Fish, J., Evans, J. J., Nimmo, M., Martin, E., Kersel, D., Bateman, A., et al. (2007).

Rehabilitation of executive dysfunction following brain injury: "content-free" cueing improves

everyday prospective memory performance. Neuropsychologia , 45(6), 1318-30.

Frueh, B. C., Monnier, J., Yim, E., Grubaugh, A. L., Hamner, M. B., & Knapp, R. G. (2007). A

randomized trial of telepsychiatry for post-traumatic stress disorder. Journal of Telemedicine and

Telecare , 13, 142-147.

Germain, V., Marchand, A., Bouchard, S., Drouin, M., & Guay, S. (2009). Effectiveness of

Cognitive Behavioural Therapy Administered by Videoconference for Posttraumatic Stress

Disorder. Cognitive Behaviour Therapy , 38(1), 42-53.

Gilbert, S. J., & Burgess, P. W. (2008). Executive Function. Current Biology , 18, R110-114.

Gorelick, P. B., Scuteri, A., Black, S. E., DeCarli, C., Greenberg, S. M., Iadecola, C., et al.

(2011). Vascular Contributions to Cognitive Impairment and Dementia : A Statement for

Healthcare Professionals From the American Heart Association/American Stroke Association.

Stroke , 42, 2672-2713.

Gowan, J. A., & Downs, J. M. (1994). Video Conferencing Human-Machine Interface.

Information and Management , 27, 341-356.

Hachinski, V., Iadecola, C., Petersen, R. C., Breteler, M. M., Nyenhuis, D. L., Black, S. E., et al.

(2006). National Institute of Neurological Disorders and Stroke–Canadian Stroke Network

Vascular Cognitive Impairment Harmonization Standards. Stroke , 37, 2220-41.

Hart, T., & Evans, J. (2006). Self-regulation and goal theories in brain injury rehabilitation. The

Journal of Head Trauma Rehabilitation , 142-155.

Heart and Stroke Foundation. (2012). Retrieved November 15, 2012, from Heart and Stroke

Foundation Website: http://www.heartandstroke.com

Heaton, R. K., Temkin, N., Dikmen, S., Avitable, N., Taylor, M. J., Marcotte, T. D., et al.

(2001). Detecting change: A comparison of three neuropsychological methods, using normal and

clinical samples. Archives of Clinical Neuropsychology , 16, 75-91.

58

Jacobson, N. S., & Truax, P. (1991). Clinical significance: a statistical approach to defining

meaningful change in psychotherapy-research. Journal of Consulting and Clinical Psychology ,

59, 12-19.

James , I. A., Blackburn, I. M., & Reichelt, F. K. (2001). Manual of the Revised Cognitive

Therapy Scale (CTS-R).

Jin, Y., Legge, S. D., Ostbye, T., Feightner, J. W., & Hachinski, V. (2006). The reciprocal risks

of stroke and cognitive impairment in an elderly population. Alzheimer's and Dementia , 171-

178.

Kennedy, M. R., Coelho, C., Turkstra, L., Ylvisaker, M., Moore Sohlberg, M., Yorkston, K., et

al. (2008). Intervention for executive functions after traumatic brain injury: a systematic review,

meta-analysis and clinical recommendations. Neuropsychological Rehabilitation , 18(3), 257-

299.

Levine, B. (1999). Methodology of frontal and executive function. Trends In Cognitive Sciences

, 3(1), 42.

Levine, B., & Downey-Lamb, M. (2002). Design and evaluation of intervention experiments. In

E. P.J. (Ed.), Neuropsychological Interventions: Clinical Research and Practice (pp. 80-104).

New York: Guilford Publications.

Levine, B., & Turner, G. R. (2004). Disorders of executive functioning and self-awareness. In

Cognitive and Behavioral Rehabilitation: From Neurobiology to Clinical Practice. Guilford

Press.

Levine, B., Freedman, M., Dawson, D., Black, S., & Stuss, D. T. (1999). Ventral frontal

contribution to self-regulation: Convergence of episodic memory and inhibition. Neurocase , 5,

263-275.

Levine, B., Manly, T., & Robertson, I. H. (2008). Goal Management Training: Administrator's

Manual. Toronto, Ontario: Baycrest Centre.

59

Levine, B., Robertson, I. H., Clare, L., Carter, G., Hong, J., Wilson, B. A., et al. (2000).

Rehabilitation of executive functioning: an experimental-clinical validation of goal management

training. Journal of International Neuropsychological Society , 6(3), 299-312.

Levine, B., Schweizer, T. A., O'Connor, C., Turner, G., Gillingham, S., Stuss, D. T., et al.

(2011). Rehabilitation of Executive Functioning in Patients with Frontal Lobe Brain Damage

with Goal Management Training. Frontiers in Human Neuroscience , 5, 9.

Levine, B., Stuss, D. T., Winocur, G., Binns, M. A., Fahy, L., Mandic, M., et al. (2007).

Cognitive rehabilitation in the elderly: effects on strategic behavior in relation to goal

management. Journal of the International Neuropsychological Society , 13(1), 143-52.

Levine, B., Turner, G. R., & Stuss, D. T. (2008). Rehabilitation of frontal lobe functions. In W.

G. Stuss D.T., Cognitive Neurorehabilitation: Evidence and Application (pp. 464-486).

Cambridge: Cambridge University Press.

Likert, R. (1932). A Technique for the Measurement of Attitudes. Archives of Psychology , 140,

1-55.

Machtmes, K., & Asher, J. W. (2000). A meta-analysis of the effectiveness of telecourses in

distance education. American Journal of Distance Education , 14(1), 27-46.

Man, D., Soong, W., Hui Chan, T., & Hui Chan, C. (2006). A randomized clinical trial study on

the effectiveness of a tele-analogy-based problem-solving programme for people withacquired

brain injury (ABI). NeuroRehabilitation , 21, 205-217.

Marziali, E. (2006). Developing Evidence for an Internet-Based Psychotherapeutic Group

Intervention. Journal of Evidence-Based Social Work , 3, 149-165.

Marziali, E. (2009). E-Health Program for Patients with Chronic Disease. Telemedicine and e-

Health , 15(2), 1-6.

Marziali, E. (2008). The Design and Evaluation of e-Health Intervention Programs for Older

Adults. eHEALTH INTERNATIONAL JOURNAL , 6-13.

60

Marziali, E., & Donahue, P. (2006). Caring for Others: Internet Video-Conferencing Group

Intervention for Family Caregivers of Older Adults With Neurodegenerative Disease. The

Gerontologist , 3, 398-403.

Marziali, E., Damianakis, T., & Donahue, P. (2006). Internet-Based Clinical Services: Virtual

Support Groups for Family Caregivers. Journal of Technology in Human Services , 24(2), 39-54.

Mattson, A. J., & Levin, H. S. (1990). Frontal lobe dysfunction following closed head injury. A

review of the literature. The Journal of Nervous and Mental Disease , 178(5), 282-291.

Moore, M., & Kearsley, G. (1996). Distance Education: A Systems View. Belmont, California:

Wadsworth.

Morland, L. A., Pierce, K., & Wong, M. Y. (2004). Telemedicine and coping skills groups for

Pacific Island veterans with post-traumatic stress disorder: A pilot study. Journal of

Telemedicine and Telecare , 10, 286-289.

Norman, D. A., & Shallice, T. (1980). Attention to Action: Willed and automatice control of

behaviour. In M. S. Gazzaniga (Ed.), Cognitive Neuroscience: A Reader (pp. 376-390). Malden,

Massachusetts: Blackwell Publishers Ltd.

Parsons, T. D., Notebaert, A. J., Shields, E. W., & Guskiewicz, K. M. (2009). Application of

Reliable Change Indices to Computerized Neuropsychological Measures of Concussion.

International Journal of Neuroscience , 119, 492-507.

Proal, E., Reiss, P. T., Klein, R. G., Mannuzza, S., Gotimer, K., Ramos-Olazagasti, M. A., et al.

(2011). Brain Gray Matter Deficits at 33-Year Follow-up in Adults With Attention-

Deficit/Hyperactivity Disorder Established in Childhood. Archives of General Psychiatry , 68

(11), 1122-1134.

Rees, L., Marshall, S., Hartridge, C., Mackie, D., Weiser, M., & Erabi Group. (2007). Cognitive

Interventions Post Acquired Brain Injury. Brain Injury , 21(2), 161-200.

Rockwood, K., Joyce, B., & Stolee, P. (1997). Use of Goal Attainment Scaling in Measuring

Clinically Important Change in Cognitive Rehbilitation Patients. Journal of Clinical

Epidemiology , 50 (5), 581-588.

61

Ruwaard, J., Broeksteeg, J., Schrieken, B., Emmelkamp, P., & Lange, A. (2010). Web-based

therapist-assisted cognitive behavioral treatment of panic symptoms: A randomized controlled

trial with a three-year follow-up. Journal of Anxiety Disorders , 24, 387-396.

Ruwaard, J., Schrieken, B., Schrijver, M., Broeksteeg, J., Kekker, J., Vermeulen, H., et al.

(2009). Standardized Web-Based Cognitive Behavioral Therapy of Mild to Moderate

Depression: A Randomized Controlled Trial with a Long-Term Follow-Up. Cognitive Behavior

Therapy , 38(4), 206-221.

Schweizer, T. A., Levine, B., Rewilak, D., O'Connor, C., Turner, G., Alexander , M. P., et al.

(2008). Rehabilitation of Executive Functioning After Focal Damage to the Cerebellum. The

American Society of Neurorehabilitation , 22, 72-77.

Schweizer, T. A., Levine, B., Rewilak, D., O'Connor, C., Turner, G., Alexander, M. P., et al.

(2008). Rehabilitation of executive functioning after focal damage to the cerebellum.

Neurorehabilitation and Neural Repair , 22, 72-77.

Shallice, T., & Burgess, P. (1996). The domain of supervisory processes and temporal

organization of behaviour. Philosophical Transactions of the Royal Society of London B , 351,

1405-1412.

Shallice, T., & Burgess, P. W. (1991). Deficits in Strategy Application Following Frontal Lobe

Damage in Man. Brain , 114, 727-741.

Stevens, J. (1996). pplied multivariate statistics for the social sciences (3rd ed.). Mahwah, New

Jersey: Lawrence Earlbaum Associates.

Stucki, G., Ewert, T., & Cieza, A. (2002). Value and application of the ICF in rehabilitation

medicine. Disability and Rehabilitation , 24(17), 932-938.

Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and

integration. Journal of the International Neuropsychological Society , 12, 261-271.

Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and

integration. Journal of the International Neuropsychological Society , 12, 261-271.

62

Stuss, D. T. (2007). Philosophical Transactions of the Royal Society B. In B. Miller, & J.

Cummings, The Human Frontal Lobes: Functions and Disorders (pp. 292-305). New York:

Guilford Press.

Stuss, D. T., & Alexander, M. P. (2000). Executive functions and the frontal lobes: a conceptual

view. Psychological Research , 63, 289-298.

Stuss, D. T., & Alexander, M. P. (2007). Is there a dysexecutive syndrome? Philosophical

Transactions of the Royal Society B , 362, 901-915.

Stuss, D. T., & Levine, B. (2002). Adult clinical neuropsychology: lessons from studies of the

frontal lobes. Annual Review of Psychology , 53, 401-433.

Sugrue, B. (2003). State of the industry: ASTD's annual review of U.S. and international trends

in workplae learning and performance. . Alexandria, VA: American Society for Training and

Development.

Turner, G. R., & Levine, B. (2008). Augmented neural activity during executive control

processing following diffuse axonal injury. Neurology , 71, 812-18.

van Hooren, S. A., Valentijn, S. A., Bosma, H., Ponds, R. W., van Boxtel, M. P., Levine, B., et

al. (2007). Effect of a structured course involving goal management training in older adults: a

randomised controlled trial. Patient Education and Counselling , 65, 205-213.

Verghese, J., Wang, C., Katz, M., Sanders, A., & Lipton, R. B. (2009). Leisure Activities and

Risk of Vascular Cognitive Impairment in Older Adults. Journal of Geriatric Psychiatry and

Neurology , 2, 110-118.

Vermeer, S. E., Prins, N. D., den Heijer, T., Hofman, A., Koudstaal, P. J., & Breteler, M. M.

(2003). Silent Brain Infarcts and the Risk of Dementia and Cognitive Decline. The New England

Journal of Medicine , 348, 1215-22.

Vrankrijker, K. (2003). The long way from the International Classification of Impairments,

Disabilities and Handicaps (ICIDH) to the International Classification of Functioning,

Disabilities and Handicaps (ICF). Disability and Rehabilitation , 25, 561-564.

63

Webster, J., & Hackley, P. (1997). Teaching effectiveness in technology-mediated distance

learning. Academy of Management Journal , 40(6) , 1282-1309.

Wechsler, D. (1985). The Wechsler adult intelligence scale- revised. New York: Psychological

Corporation.

West, R., Ergis, A. M., Winocur, G., & Saint-Cyr, J. (1998). The Contribution of Impaired

Working Memory Monitoring to Performance of the Self-Ordered Pointing Task in Normal

Aging and Parkinson's Disease. Neuropsychology , 12, 546-554.

WHO. (2001). International Classification Functioning, Disability and Health (ICF). Geneva:

World Health Organization.

64

Appendix A: Cognitive Rehabilitation Therapy Scale for Goal Management Training – (CRTS-GMT)

65

66

67

68

69

70

71

72

73

74

75

Appendix B: Participant Feedback Survey

Web-Based BTP- Participants Questionnaire

Please answer the following questions based on your experience as a participant in the Brain

Training Program Study.

1) Overall how easy or difficult did you find the use of Adobe Connect?

a. Very easy

b. Somewhat easy

c. Neutral

d. Somewhat difficult

e. Very difficult

2) Overall how helpful was the technical support offered (i.e manual provided and

Illia/Halla’s help)?

a. Very helpful

b. Somewhat helpful

c. Not very helpful

d. Not helpful at all

e. I didn’t need technical support

3) How helpful did you find adobe connect specifically in the learning experience?

a. Very helpful

b. Somewhat helpful

c. Not very helpful

d. Not helpful at all, I would rather not use it for this program

4) If you were to participate in this brain training program again, would you rather sign up

to:

a. A web-based group

b. An in-class group

What are some things that can be done to improve the online delivery experience?

Additional comments:

76

Appendix C: WebGMT Groups 1 and 2 compared to In-Class GMT on CRTS-GMT scale

Session 5 9

In-Class (IC) vs. Web-Based (WB) sessions IC WB1 WB2 IC WB1 WB2

1. Adherence to module outline 6 4 6 6 4.5 6

2. Feedback 5 6 6 5 6 5.5

3. Collaboration (verbal and non-verbal) 5 3 6 4.5 5 5

4. Pacing and efficient use of time 6 5 6 6 5 6

5. Interpersonal effectiveness 5 6 6 4.5 6 6

6. Conceptual integration 5 6 6 4.5 6 6

7. Therapist Skills 5 5 6 4.5 6 6

8. Homework Setting 6 5 6 n/a 6

9. Participant Engagement 5.5 5 6 5 5.5 5

Total (max. 54) 48 45 54 40 50 45.5

Comparing WebGMT to In-Class Group 1 Group 2

Percent of times Web-Based scores equal or

higher than In-Class session

55% 83%