1 Classifying Instantaneous Cognitive States from fMRI Data Tom Mitchell, Rebecca Hutchinson, Marcel...
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
0
Transcript of 1 Classifying Instantaneous Cognitive States from fMRI Data Tom Mitchell, Rebecca Hutchinson, Marcel...
1
Classifying Instantaneous Cognitive States from fMRI Data
Tom Mitchell, Rebecca Hutchinson, Marcel Just, Stefan Niculescu, Francisco Pereira, Xuerui Wang
Carnegie Mellon University
November, 2003
2
…
3
4
Cognitive state sequence
COGNITIVE TASK
“Virtual sensors” of cognitive state
1. Does fMRI contain enough information?
2. Can we devise learning algorithms to construct such “virtual sensors”?
5
Learning Virtual Sensors
• Learn fMRI(t,t+k) CognitiveState
• Classifiers:– Gaussian Naïve Bayes, SVM, kNN
• Trained per subject, per experiment
• Feature selection/abstraction– Select subset of voxels (by signal, by anatomy)– Select subinterval of time– Average activities over space, time– Normalize voxel activities
6
Study 1: Pictures and Sentences
• Trial: read sentence, view picture, answer whether
sentence describes picture
• Picture presented first in half of trials, sentence first
in other half
• Three possible objects: star, dollar, plus
• Collected by Just et al.
7
It is true that the star is above the plus?
8
9
+
---
*
10
11
Is Subject Viewing Picture or Sentence?
• Learn fMRI(t,t+8) {Picture, Sentence}
• Leave two out cross-validation was used to assess the performance of the classifiers
• SVMs and GNB worked better than kNN
• Some Details: – 12 subjects, 40 pictures, 40 sentences– 1397 - 2864 voxels per subject, 7 ROIs – fMRI snapshot taken every half second
12
Error for Single-Subject Classifiers
•Error computed by averaging over all subjects
•95% confidence intervals per subject are ~ 10% large
• Error of default classifier is 50%
Dataset \ Classifier GNB SVM 1NN 2NN 5NN
Picture vs Sentence 0.16 0.09 0.20 0.18 0.19
13
• Approach: define supervoxels based on anatomically defined regions of interest– Normalize per voxel activity for each subject
• Each value scaled now in [0,1]
– Abstract to seven brain region supervoxels– 16 snapshots for each supervoxel
• Train on n-1 subjects, test on nth– Leave one subject out cross validation
Can We Train Subject-Indep Classifiers?
14
• NO Feature Selection used in this experiment
•95% confidence intervals approximately 5% large
•Error of default classifier is 50%
Error for Cross Subject Classifiers
Dataset \ Classifier GNB SVM 1NN 2NN 5NN
Cross-Subject Pict vs Sent
0.30 0.25 0.36 0.33 0.32
15
Study 2: Word Categories
• Family members
• Occupations
• Tools
• Kitchen items
• Dwellings
• Building parts
• 4 legged animals
• Fish
• Trees
• Flowers
• Fruits
• Vegetables
16
Word Categories Study
• Stimulus:
– 12 blocks of words:
• Category name (2 sec)
• Word (400 msec), Blank screen (1200 msec); answer
• Word (400 msec), Blank screen (1200 msec); answer
• …
– Subject answers whether each word in category
– 20 words per block, nearly all in category
17
Training Classifier for Word Categories
• Learn fMRI(t) Word Category
• Training methods: kNN, GNB
• Leave one example out from each class used to assess performance
• Some Details: – 10 subjects, 20 examples per class– 8470 - 11,136 voxels per subject, 30 ROIs– fMRI snapshot taken every second
18
Study 2: Results
Classifier outputs ranked list of classesEvaluate by the fraction of classes ranked ahead of true
class– 0=perfect, 0.5=random, 1.0 unbelievably poor
Dataset \ Classifier
GNB 1NN 3NN 5NN
Words 0.08 0.30 0.20 0.16
19
Study 3: Syntactic Ambiguity
• Is subject reading ambiguous or unambiguous sentence?– “The experienced soldiers warned about the dangers
conducted the midnight raid.”
– “The experienced soldiers spoke about the dangers before the midnight raid.”
• Almost random results if no feature selection used • With feature selection:
– SVM - 77% accuracy
– GNB - 75% accuracy
– 5NN – 72% accuracy
20
• Four feature selection methods:
• Active (n most active available voxels compared to baseline fixation activity, according to a t-test)
• RoiActive (n most active voxels in each ROI)
• RoiActiveAvg (average of the n most active voxels in each ROI)
• Disc (n most discriminating voxels according to a trained classifier)
• Active works best
Feature Selection
21
Dataset
Feature
Selection
GNB SVM 1NN 3NN 5NN
PictureSent
No 0.29 0.32 0.43 0.41 0.37
Active 0.16 0.09 0.20 0.18 0.19
Words
No 0.10 N/A 0.40 0.40 0.40
Active 0.08 N/A 0.30 0.20 0.16
SyntAmb
No 0.43 0.38 0.50 0.46 0.47
Active 0.25 0.23 0.29 0.29 0.28
Feature Selection
22
Summary
• Proved that there is enough information in the fMRI signal to allow decoding of Cognitive States
• Successful training of classifiers for instantaneous cognitive state in three studies
• Cross subject classifiers trained by abstracting to anatomically defined ROIs
• Feature selection and abstraction are essential
23
Research Opportunities
• Learning temporal models– HMM’s, Temporal Bayes Nets
• Learn to discriminate whether a subject has certain mental disease
• Discovering useful data abstractions– ICA, PCA, hidden layers in Neural Nets
• Merging data from multiple sources– fMRI, ERP, reaction times