Classification and Clustering of Brain Pathologies from Motion

Data of Patientsin a Virtual Reality Environment

Via Machine Learning

Uri Feintuch, Hadassah- Hebrew University Medical Center

Larry Manevitz, University of Haifa,

Natan Silnitsky, University of Haifa

Data from: Assaf Dvorkin, Northwestern University

BISFAI, June 2011

Brain pathologies

• CVA - CerebroVascular Accident (Stroke)

– Hemispatial neglect

• TBI - Traumatic Brain Injury

Rehabilitation

• Diagnosis– Differential Diagnosis (e.g., Neglect vs.

Hemianopsia)

• Evaluation– Severity of deficit– Progress during intervention

Example - Neglect

(Deouell, Sacher & Soroker, 2005)

• HD applied for and received back his driver’s license, having shown intact visual fields at Perimetry and no signs of neglect.

• HD scored 143/146 on his BIT test.

• Since obtaining the license, however, he was involved in 9 car accidents, all concerning the left side of his car.

and their shortcomings…

VR in Rehab (1)

• Virtual Mall

A camera which films the patient and a monitor which displays her

The way the patient views himself within the virtual environment

VR in Rehab

• Replaces traditional methods

• Ecological validity

• Safety

• Absolute control of stimuli

VR in Rehab (2)

• Assaf Dvorkin, Rehabilitation Institute of Chicago, Northwestern Uni.

The VRROOM haptics/graphic system A target in the field of view

VR in Rehab - Challenges

• Human behavior is very complicated

• Vast amount of information

• geometry or physics formula (?)

• Simplistic analysis (e.g., RT, % Errors)

Research Goals

• Identify and differentiate between meaningful clinical conditions – Scarce data– Perhaps noisy

• Broad spectrum conditions like neglect– Mild, severe– Use unsupervised learning approach

Machine Learning Techniques Used

• Supervised Learning– Backpropogation NN– SVM

• Unsupervised Learning– Kohonen– K-means

2D Experiment

• Population: 54 HA, 11 CVA (without neglect), 9 TBI, 25 HC.

• Data Encoding: Vector of hand movement (dx,dy,dt)

NN Architecture for 2D

dx dy dt dx dy dt dx dy dt dx dy dt dx dy dt

…(Full connectivity)…

Data point (t) Data point (t+1) Data point (t+2) Data point (t+3) Data point (t+4)

Input Layer

Hidden Layer

Output Layer

NN Architecture for 2D (TBI vs. CVA)

dx

…(Full connectivity)…

Data point (t) Data point (t+1) Data point (t+2) Data point (t+3) Data point (t+4)

Input Layer

Output Layer

dy dt dx dy dt dx dy dt dx dy dt dx dy dt

…(Full connectivity)…

Training for 2D

• Levenberg-Marquardt

• resilient back-propagation

• 300 epochs

• Cross-validation

2D Results – 2 class

PopulationsBP NN

Average Success(by subjects)

Healthy/CVA90%

Healthy/TBI100%

TBI/CVA97%

3D Experiment

• Population: 9 H, 9 N, 10 S• Data Encoding: Vector of movement

(x,y,z,t)• Only trials where movement occurred at all• Phases

– Long vector: Entire trial from appearance of stimulus (includes pre-movement data)

– Movement: Vector only from commencement of movement

– Initial/Final segment – beginning/end of movement

NN Architecture for 3D

• 1400 elements for a long vector (1400-5-1)

• 1000 elements for a movement vector (1000-5-1)

• 130 elements for initial/final vectors (130-5-1)

3D Data Set

• Population of Healthy, Neglect, Stroke

• Movement Vectors (x,y,z,t) of different lengths

• Also tested on “snippets” for cross platform

• Resilient back-propagation

• 50 to 300 epochs

3D Results – 2 classVector sizePopulations BP NN Success Rate

By Patient and

By Data Point Including Reaction Time (470 time steps)

Healthy/CVA78%

62% (4010/6458)

Healthy/Neglect89%

79% (4557/5791)

Neglect/CVA72%

63% (3584/5703)

From Start of Motion

(330 time steps)

Healthy/CVA78%

68% (4378/6458)

Healthy/Neglect89%

78% (4515/5791)

Neglect/CVA72%

66% (3737/5703)

3D Results – 2 classVector sizePopulations BP NN Success Rate

By Patient and

By Data Point Small initial segment

(43 time steps)

Healthy/CVA39%

43% (2791/6458)

Healthy/Neglect83%

70% (4058/5791)

Neglect/CVA78%

61% (3470/5703)

Small final segment

(43 time steps)

Healthy/CVA67%

53% (3435/6458)

Healthy/Neglect89%

61% (3547/5791)

Neglect/CVA61%

52% (2962/5703)

Clustering for 3D

• Kohonen Self Organizational Map (SOM)

• Reproduced with K-means

Clustering for 3D

Clustering for 3D • 2 Neurons

• 7 Neurons

• 200 Neurons

3D Results – 0 class• Movement Vector, Neglect, (7 clusters)

• Movement, Healthy/Neglect, (7)

3D Results – 0 class• Movement Vector, Neglect/CVA ,

(7 clusters)

Clustering for 2D

• Kohonen Self Organizational Map (SOM)

• Reproduced with K-means

2D Results – 0 class• Healthy/CVA, (7 clusters)

• -> …

3D expriment - 1 class

• Architecture

• Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130

3D expriment - 1 class

• Architecture

• Movement vectors – 1000-200-1000 • initial/final vectors - 130-26-130

3D expriment - 1 class• Threshold

choice

1 class results for 3D

Vector sizeTraining Set Population

Compression NNAverage Success

From Start of Motion

(330 time steps) Healthy 93%

CVA 69%

Neglect 50% Non-Mild Neglect76%

Severe Neglect 100%

1 class results for 3D

• Neglect classifier for "Left targets trials only" - 62%

• Non-Mild Neglect classifier for "Left targets trials only" - 83%

Combined Platforms• Merging small samples from different

platforms. But…

Combined Platforms – 2 class

• (x,y,z) -> (x,y,0)

• "snippets"

• Experiments different data amounts

Combined Platforms – 2 class

Vector sizeTraining Set Origin

BP NNAverage Success

30 time steps VRROOM and GestureXtreme

75%

90 time steps 90%

VRROOM only 50%

GestureXtreme only

50%

Summary of results

• 2D experiment – Differential Diagnosis:– Healthy vs. Patients– TBI vs. CVA

• 3D experiment – DD + Evaluation – Neglect vs CVA– Clusters by severity– 1 class classifiers (Severe Neglect)

Future Work

• Merging data across platforms

• Automatic Prognosis and Individualized Treatment Protocols

– Construct models of patients with their movement restrictions

– Run potential rehab protocols on the model– Prognosis: via best results on model– Apply best protocol to the patient

Acknowledgment

• Assaf Dvorkin • Jim Patton • Eugene Mednikov • Debbie Rand • Rachel Kizony • Neta Erez • Meir Shahar • Patrice L. Weiss • The Caesarea Rothschild Institute