Automated Detection of Change in Serial MR Brain Studies Julia Patriarche, Ph.D. November 18, 2008.
-
date post
21-Dec-2015 -
Category
Documents
-
view
214 -
download
1
Transcript of Automated Detection of Change in Serial MR Brain Studies Julia Patriarche, Ph.D. November 18, 2008.
Automated Detection of Change in Serial MR Brain Studies
Julia Patriarche, Ph.D.
November 18, 2008
Brain Tumor Suspected
Brain Tumor Suspected
Neurologic exam, Imaging, Biopsy
Resection, radiation, chemotherapy
Neurologic exam, Imaging, Biopsy
Brain Tumor Suspected
Resection, radiation, chemotherapy
Serial Imaging
Neurologic exam, Imaging, Biopsy
Brain Tumor Suspected
Baseline Followup
Change
or No Change?
FollowupBaseline
FollowupBaseline
Scanner has changedScanner Software has changedPulse Sequences have changed
Acquisition parameters have changedGradient Coils have changed
RF Inhomogeneities have changed Registration has changed
…
Change Detection is not so simple
1. Separation of acquisition changes from disease related changes 2. Information overload
3. Change Blindness / Inattentional Blindness
Change Blindness
Our visual systems have evolved to control our sensitivity to change
What we see is in the moment is richly detailed, but what is held in memory from one moment to the next is not the detailed visual information – it is abstracted.
Mirror Example
Ronald A. Rensink, University of British Columbia http://www.psych.ubc.ca/~rensink/flicker/download/
Changing case of letters
McConkie, G.W. and Zola, D. “Is visual information integrated across successive fixations in reading?” Percept. Psychophysiol. 1979; 25: 221-224.
AITeRnAtEd CaSe
Changing case of letters
McConkie, G.W. and Zola, D. “Is visual information integrated across successive fixations in reading?” Percept. Psychophysiol. 1979; 25: 221-224.
AITeRnAtEd CaSe
altErNaTeD cAsE
Change Blindness
There really isn’t a mechanism for changes in this abstracted information to grab your attention.
Change Blindness
There really isn’t a mechanism for changes in this abstracted information to grab your attention.
If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll
miss changes.
Change Blindness
There really isn’t a mechanism for changes in this abstracted information to grab your attention.
If you don’t specifically attend to some feature, or some aspect of that feature at both time-points and then consciously compare them, there is a very good chance you’ll
miss changes.
Trying to find subtle changes, in extent, or character is very difficult.
Change Blindness
There is a whole area of study devoted to change blindness
Change Blindness
There is a whole area of study devoted to change blindness
People tend not to be aware of the extent of change blindness (i.e. change blindness-blindness is a topic of study)
Change Blindness
There is a whole area of study devoted to change blindness
People tend not to be aware of the extent of change blindness (i.e. change blindness-blindness is a topic of study)
Visual change detection is so difficult that there are even games where you’re supposed to find the differences.
Change Blindness
The point: side-by-side presentation is very poorly matched to the human visual system. Unregistered images, inhomogeneity effects, information overload make
this problem even more challenging.
Change
or No Change?
Baseline Follow-up
Other reasons radiologists might miss changes:
Satisfaction of SearchChange occurs in an unexpected location
Change is in a sub-part of a complicated lesionEtc.
Not surprisingly…
Change detection is thought to be not very reproducible
What is desired:
1. Systems which reduce the quantity of data presented to the radiologist,
2. Systems which help to separate acquisition related change from disease related change,
3. Systems which present data in a way matched to the human visual system
4. Systems which produce objective, reproducible, and accurate metrics
Something which draws the radiologist’s attention to those changes.
Something which presents what is of interest – changes – to the radiologist.
It’s easier to understand the nature of the changes as a whole, if you can view an image of the changes.
Change Detector System Flow
Inhomogeneity Correction
Registration
Change Detector
Presentation of Results
Image Acquisition Consistent With Objective
Acquisition Considerations
To the Greatest Degree Possible:
Reduce slice thicknessEliminate inter-slice gaps
3D acquisitionEquivalent scanner
Equivalent scanner softwareEquivalent pulse sequences
Equivalent acquisition parametersCorrect for decay of gradient coils
Correct for inhomogenetiesIdentical administration of contrast
As with an experiment,
Control the variables, to isolate the changes of interest
Change Detector Algorithm Organization
Define Imaging Properties for: NAWM, Gray Matter, CSF,Lesion, and Enhancing Lesion
Define Brain Parenchyma
Compute Lesion and Enhancing Lesion Memberships On aVoxel-by-voxel Basis
Significant Region Detection
Automated Sample PointsGeneration / Automated
Parenchyma MaskGeneration
Lesion Finder
Noise Reduction
Change Detector Compute Measures of Change On a Voxel-by-voxel Basis,expressed in Δ membership of lesion and enhancing lesion
The algorithm is designed to detect changes which are:
Subtle in extentSubtle in degree
The algorithm produces both:
Quantitative summariesQualitative color change maps
Baseline
Followup
Baseline Followup
Color Change Map
Original Clinical Interpretation: Stable / Time to Progression
Diagnosis = 7 months
Baseline Followup
Baseline Followup
Color Change Map
Original Clinical Interpretation: Stable / Time to Progression
Diagnosis = 3.25 months
Baseline Followup
Baseline Followup
Color Change Map
Original Clinical Interpretation: Stable / Time to Progression
Diagnosis = 4.75 months
Baseline Follow-up
Baseline Follow-up
Color Change Map
Original Clinical Interpretation: Stable / Time to Progression
Diagnosis > 18 months
Determining that something hasn’t changed, can be as
tricky as determining that it has
Study #1: Assisted Serial MRI Examination
Sensitivity: Fraction of the actually positive cases which were correctly identified
Specificity: Fraction of the actually negative cases which were correctly identified
Study #1: Assisted Serial MRI Examination
•6 neuroradiologists examined a series of 50 serial MR pairs from 28 patients•Asked to rate each case as stable or progressing•Double cross over study with one month separation
Study #1 Results
• All but one neuroradiologist had improved sensitivity
• Two NRs achieved perfect sensitivity with the change detector, none without
• Mean accuracy improved from 0.859 to 0.891
• Mean sensitivity was largely unchanged, from 0.892 to 0.891
• Mean specificity improved moderately from 0.848 to 0.874
Interobserver Agreement
• Without the change detector, there were 17 cases in which all the neuroradiologists agreed with the gold standard
• With the change detector, there were 23
Study #1 Results:
• Automated system examined 88 serial comparisons• Divided cases into: stable or progressing
• Automated diagnosis was compared to original clinical diagnosis
Study #2: Automated Serial MRI Examination
Suggests that the change detector may be able to detect changes earlier than expert neuroradiologists.
Patriarche JW, Erickson BJ. "Part 2. Automated Change Detection and Characterization Applied to Serial MR of Brain Tumors May Detect Progression Earlier Than Human Experts". Journal of Digital Imaging 2007; 20(4): 321-328.
Reasons for good outcome:
• No objective definition of progression.
Reasons for good outcome:
• No objective definition of progression.
• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
Reasons for good outcome:
• No objective definition of progression.
• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
• Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences
Reasons for good outcome:
• No objective definition of progression.
• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
• Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences
• Changes unexpected in location, etc.
Reasons for good outcome:
• No objective definition of progression.
• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
• Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences
• Changes unexpected in location, etc.
• Satisfaction of Search
Reasons for good outcome:
• No objective definition of progression.
• Subtle changes difficult to see • Especially when images displayed side-by-side • Especially when images not registered
• Changes may not be obvious looking at only one pulse sequence, but might present across pulse sequences
• Changes unexpected in location, etc.
• Satisfaction of Search
• Information overload - impractical or impossible to wade through every possible comparison manually
The Point:
• Computers are great at methodically wading through large amounts of data, looking for needles in hay-stacks, integrating large amounts of data at once, applying serial processing steps, analyzing data mathematically, and bringing noteworthy observations to the attention of the neuroradiologist.
Eric Kischell has implemented a very streamlined workflow• Provides full automation, end-to-end
Liqin Wang has implemented a web-based interface • Neuroradiologists can review change detector output from any computer with a web-browser• Allows sophisticated interaction with output, e.g. via two color map modes, linked cursors, and ‘flicker mode’ viewing of serial images.
Current Work
Clinical Deployment:
• Facilitate studies with more patients and neuroradiologists• Facilitate more varied study designs• Directly aid patients
Other Developments:
• Full automation / full reproducibility - removes barriers to widespread deployment.
Other Developments:
• Full automation / full reproducibility - removes barriers to widespread deployment.
• Greatly improved sensitivity and specificity
Layered AI Architecture:
Change detector possesses multiple layers of automated reasoning / information gathering.
Starts with simplistic but highly reliable knowledge, and uses this to acquire case-specific information.
Uses this case-specific information to acquire more complex information, etc. up the hierarchy.
SignificantRegion Detector
SignificantRegion Detector
AutomatedSample Points Algorithm
Parenchyma MaskAlgorithm
SignificantRegion Detector
AutomatedSample Points Algorithm
Parenchyma MaskAlgorithm
Lesion Finder
SignificantRegion Detector
AutomatedSample Points Algorithm
Parenchyma MaskAlgorithm
Lesion Finder
Change Detector
SignificantRegion Detector
AutomatedSample Points Algorithm
Parenchyma MaskAlgorithm
Lesion Finder
Change Detector
?
SignificantRegion Detector
AutomatedSample Points Algorithm
Parenchyma MaskAlgorithm
Lesion Finder
Change Detector
?
?
Components are modular and interchangeable.
Obviously:If everything is built on the lower levels – if the reasoning is based on the knowledge gleaned by those lower levels -- those levels must produce very reliable information.
Information Overload:
• The phenomenon is extremely wide-spread.
• The problem / opportunity will almost definitely continue to grow.
Quantitative & Automated Analyses:
• Makes information overload, information affluence
Change Detector:
• May help to train new practitioners
• May serve as a model of how computer can liberate neuroradiologist from doing mundane things sub-optimally, and instead have computer wade through mountains of analysis and present the clinician with refined and concise data tailored to their purpose.
• May serve as a model of how computer can perform a mountain of peripheral analyses and bring to clinician’s attention things they weren’t looking for (but which are important).
• May serve as a model of ‘layered analyses’ which are impractical or impossible without computer assistance.
All of this:
Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.
All of this:
Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.
Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure.
All of this:
Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.
Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure.
Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it.
All of this:
Designed to translate what is acquired, into what is desired – data tailored to the task at hand, by using processing and domain specific knowledge.
Don’t ask the radiologist to do the routine / protocol-driven work – define the procedure, and build a machine to perform that procedure.
Use the computer’s ability to do routine pre-defined tasks precisely, every time, and to handle complex quantitative and layered analyses with ease; to do massive numbers of procedure-driven analyses on each case, to present highly processed information tailored to the task at hand, and to direct attention to identified features which require it.
Use the computer as a smart helper, who goes away and does tons of analysis, and who brings to the attention of the radiologist things which should be attended to.
Some clinical reasons the change detector is of value:
Create a quantitative definition of progression
Possible improved outcomes through earlier interventions
Potentially facilitate screening
Potentially facilitate personalized therapy
Compare novel therapies, through automatic, reproducible, and quantitative measures of change.
Future Directions:
Many possibilities.
Future Directions:
Many possibilities.
Add additional pulse sequences (e.g. diffusion, perfusion, etc.).
Future Directions:
Many possibilities.
Add additional pulse sequences (e.g. diffusion, perfusion, etc.).
Add more complex anatomical knowledge (e.g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.
Future Directions:
Many possibilities.
Add additional pulse sequences (e.g. diffusion, perfusion, etc.).
Add more complex anatomical knowledge (e.g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.
Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy.
Future Directions:
Many possibilities.
Add additional pulse sequences (e.g. diffusion, perfusion, etc.).
Add more complex anatomical knowledge (e.g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.
Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy.
Apply technique to different anatomical regions (e.g. breast, etc.)
Future Directions:
Many possibilities.
Add additional pulse sequences (e.g. diffusion, perfusion, etc.).
Add more complex anatomical knowledge (e.g. so change detector knows what is white matter, gray matter, CSF, and can act appropriately.
Identify and quantify different kinds of changes, ex. attempt to differentiate between character change and mass effect; identify and quantify atrophy.
Apply technique to different anatomical regions (e.g. breast, etc.)
A variety of clinical studies.
Acknowledgements
Christopher Wood, M.D.
Norbert Campeau, M.D.
Edward Lindell, M.D.
Vladimir Savcenko, M.D.
Norman Arslanlar, D.O.
Bradley Erickson M.D., Ph.D.
Acknowledgements
Bradley Erickson, M.D., Ph.D. – Director, Mayo Clinic Radiology Informatics Lab
Brian O’Neill, M.D. – Director, Mayo Neurology Fellowship Program
Christopher Chute, M.D., Dr. P.H. – Director, Mayo Informatics Fellowship Program
Eric Kischell – Programmer, implemented change detector integrated system
Liqin Wang – Programmer, implemented registration algorithm, web interface
Mayo Brain Computation Workgroup
University of Hawaii – Department of Information and Computer Sciences
NIH T32 NS07494-03
NIH T15 LM07041-23