Classification and Clustering of Brain Pathologies from Motion Data of Patientsin a Virtual Reality EnvironmentVia Machine LearningUri Feintuch, Hadassah- Hebrew University Medical Center Larry Manevitz, University of Haifa,Natan Silnitsky, University of Haifa Data from: Assaf Dvorkin, Northwestern University Neuro computation laboratory day, December 2011

OutlineRehabilitation of Patients of Brain PathologiesVirtual Reality (VR) in Rehab Research GoalsTechniques UsedExperimentsArchitecture and TrainingResultsFuture directions

Brain pathologiesCVA - CerebroVascular Accident (Stroke) Hemispatial neglectTBI - Traumatic Brain Injury

RehabilitationDiagnosisDifferential Diagnosis (e.g., Neglect vs. Hemianopsia) EvaluationSeverity of deficitProgress during intervention

Example - Neglect

Star CancellationTraditional tools

(Deouell, Sacher & Soroker, 2005)HD applied for and received back his drivers license, having shown intact visual fields at Perimetry and no signs of neglect.

HD scored 143/146 on his BIT test.

Since obtaining the license, however, he was involved in 9 car accidents, all concerning the left side of his car.and their shortcomings

VR in Rehab (1)Virtual Mall A camera which films the patient and a monitor which displays her The way the patient views himself within the virtual environment

VR in RehabReplaces traditional methodsEcological validitySafety Absolute control of stimuli

VR in Rehab (2)Assaf Dvorkin, Rehabilitation Institute of Chicago, Northwestern Uni. The VRROOM haptics/graphic systemA target in the field of view

VR in Rehab - ChallengesHuman behavior is very complicatedVast amount of informationgeometry or physics formula (?)Simplistic analysis (e.g., RT, % Errors)

Proposed solutionApply Machine Learning tools. Such tools may detect patterns of behavior performed within the Virtual Environment.In this work we used Artificial Neural Networks (NN) Classifiers ,SVM, SOM and k-means.

Research GoalsIdentify and differentiate between meaningful clinical conditions Scarce dataPerhaps noisy

Broad spectrum conditions like neglectMild, severeUse unsupervised learning approach

Machine Learning Techniques UsedSupervised LearningBackpropogation NNSVMUnsupervised LearningKohonenK-means

2D ExperimentPopulation: 54 HA, 11 CVA (without neglect), 9 TBI, 25 HC. Data Encoding: Vector of hand movement (dx,dy,dt)

NN Architecture for 2Ddxdydtdxdydtdxdydtdxdydtdxdydt(Full connectivity)Data point (t)Data point (t+1)Data point (t+2)Data point (t+3)Data point (t+4)Input LayerHidden LayerOutput Layer

NN Architecture for 2D (TBI vs. CVA)(Full connectivity)Data point (t)Data point (t+1)Data point (t+2)Data point (t+3)Data point (t+4)Input LayerOutput Layer(Full connectivity)

Training for 2D Levenberg-Marquardt resilient back-propagation 300 epochs Cross-validation

2D Results 2 class

3D ExperimentPopulation: 9 H, 9 N, 10 SData Encoding: Vector of movement (x,y,z,t)Only trials where movement occurred at allPhasesLong vector: Entire trial from appearance of stimulus (includes pre-movement data)Movement: Vector only from commencement of movement Initial/Final segment beginning/end of movement

NN Architecture for 3D 1400 elements for a long vector (1400-5-1)1000 elements for a movement vector (1000-5-1) 130 elements for initial/final vectors (130-5-1)

3D Data Set Population of Healthy, Neglect, StrokeMovement Vectors (x,y,z,t) of different lengths Also tested on snippets for cross platformResilient back-propagation 50 to 300 epochs

3D Results 2 class

3D Results 2 class

Clustering for 3D Kohonen Self Organizational Map (SOM)

Reproduced with K-means

Clustering for 3D

Clustering for 3D 2 Neurons

7 Neurons

200 Neurons

3D Results 0 classMovement Vector, Neglect, (7 clusters)

Movement, Healthy/Neglect, (7)

3D Results 0 classMovement Vector, Neglect/CVA , (7 clusters)

Clustering for 2D Kohonen Self Organizational Map (SOM)

Reproduced with K-means

2D Results 0 classHealthy/CVA, (7 clusters)->

3D expriment - 1 classArchitecture

Movement vectors 1000-200-1000 initial/final vectors - 130-26-130

3D expriment - 1 classArchitecture

Movement vectors 1000-200-1000 initial/final vectors - 130-26-130

3D expriment - 1 classThreshold choice

1 class results for 3D

1 class results for 3DNeglect classifier for "Left targets trials only" - 62% Non-Mild Neglect classifier for "Left targets trials only" - 83%

Combined PlatformsMerging small samples from different platforms. But

Combined Platforms 2 class(x,y,z) -> (x,y,0)"snippets" Experiments different data amounts

Combined Platforms 2 class

Summary of results2D experiment Differential Diagnosis:Healthy vs. PatientsTBI vs. CVA3D experiment DD + Evaluation Neglect vs CVAClusters by severity1 class classifiers (Severe Neglect)

Future WorkMerging data across platforms

Automatic Prognosis and Individualized Treatment ProtocolsConstruct models of patients with their movement restrictionsRun potential rehab protocols on the modelPrognosis: via best results on modelApply best protocol to the patient

AcknowledgmentAssaf Dvorkin Jim Patton Eugene Mednikov Debbie Rand Rachel Kizony Neta Erez Meir Shahar Patrice L. Weiss The Caesarea Rothschild Institute

TBI someone falls down a flight of stairs, injured in a bullfightHemianopsia Cortical Blindness*Red glove easy to detect, very noisy visual detectionIts hard to get real human data!Dx, dy, - from torso to palm of handFlow from the bottom to the top. The output layer has 1 object decides 1,-1, stroke, healthyTotal data points for 94 subjects 68,000

Total data points for 94 subjects 68,000Training was done with LM,RP, 300 epcohs.The result being presented is after cross validation

LM tricks to compute 2nd derivative faster a lot of memoryRP only sign of derivative is considered

Trials 9700 data points, 300 trials per subjectsNeglect harder to determine course at the beginning of movementSmall samples enables more systems -> more dataModel NN trained on patient (input sys params,trajectory motion)Then train on different motion requirementPrognosis model improves -> patient also