Detection and Assessment of Abnormality in Medical Images MS Thesis Presentation Candidate: K Sai...
-
Upload
rodger-beasley -
Category
Documents
-
view
215 -
download
2
Transcript of Detection and Assessment of Abnormality in Medical Images MS Thesis Presentation Candidate: K Sai...
Detection and Assessment of Abnormality in Medical Images
MS Thesis PresentationCandidate: K Sai Deepak
Adviser: Prof. Jayanthi Sivaswamy
Center for Visual Information TechnologyIIIT Hyderabad
India
31-March-2012
Agenda
• Computer Aided Diagnosis– Modes of Healthcare– CAD in Primary Care (examples)
• Disease Screening– CAD in Disease Screening– Challenges for existing CAD
• Proposed Methodology– Detecting Abnormality Instead of Disease– Detection of Lesions using Motion Patterns
• Detection and Assessment of Retinopathy– Diabetic Macular Edema– Method– Experiments and Results– Detection of Multiple Lesions
• Classification of Lesions in Mammograms– Mammographic Lesions– Experiments and Results
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Source of all the figures are explicitly mentioned in the MS Thesis
PART I – Computer Aided Diagnosis
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Computer Aided Diagnosis (CAD)
• Aid of computers in the process of diagnosis
• Computer aided diagnosis (CAD) has become one of the major support systems assisting medical experts in diagnosis through images
• CAD tools are used for measurement, display and analysis of both the structural and functional aspects of the body from images
Computer Aided Diagnosis
Computer Aided Diagnosis
CAD with Images
Computer Aided Diagnosis
Computer Aided Diagnosis
• Visualization – enhancement for visual analysis (Ex. Windowing, MIP, MAP, AIP, Zoom, Contrast Inversion etc.)
• Detection – detect the presence of disease manifestation
• Localization and Segmentation – Localize or segment the spatial regions containing disease manifestation
• Other utilities can be used for measurement of various structures from images (length, volume etc. )
Healthcare – Primary Care and Disease Screening
Computer Aided Diagnosis
Computer Aided Diagnosis
Secondary and Tertiary Care Centers – are where patients usually visit on referral for advanced care
Point of Consultation in basic healthcarePatients with Symptoms arrive
Undergo specialized tests if required for DiagnosisTreatment is planned based on Diagnosis
Performed on Public health initiativeMost patients have no disease symptoms
Detection is performed by a trained professionalReferred to expert on positive detection
CAD in Primary Care
Computer Aided Diagnosis
Computer Aided Diagnosis
• Traditionally CAD has been used in Primary Care
CAD in Primary Care
• Patient visits the doctor with a complaint
• If required, the patient is then referred by the doctor for specific imaging in order to diagnose the problem
• Acquired images are analyzed by the experts (Ophthalmologist, Radiologist) to arrive at a diagnosis
• The diagnosis report is used by doctor for planning treatment
Computer Aided Diagnosis
Computer Aided Diagnosis
PART II – Disease Screening
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Disease Screening
• Disease screening is performed at specific community healthcare centers to prevent ensuing mortality and suffering from chronic ailments
• Challenges: Geographical reach, Disease awareness and Social barriers and Availability of experts are common in screening
• Tele-radiology provides significant help but the work load of a medical expert increases significantly due to large number of patients participating in population screening
• Diabetic Retinopathy and Breast Cancer screening are already conducted or being adopted in several countries and is the focus of this work
Disease ScreeningDisease Screening
CAD in Disease Screening
• Existing CAD tools use a disease centric approach for disease detection
• It requires application of several methods/tools for detecting all the possible lesions in a disease– Multiple CAD tools are used for identifying different Diabetic
Retinopathy (DR) manifestations
• Existing CAD systems are not able to meet the needs of disease screening in Diabetic Retinopathy [1]– Poor sensitivity of disease detection – Large number of normal patients are detected as abnormal
Disease ScreeningDisease Screening
[1] M. D. Abramoff, M. Niemeijer, M. S. Suttorp-Schulten, M. A. Viergever, S. R. Russell, and B. van Ginneken. Evaluation of a system for automatic detection of diabetic retinopathy from color fundus photographs in a large population of patients with diabetes. Journal of Diabetes Care, 31:193–198, 2007.
Summary of Challenges
• Existing CAD tools use a disease centric approach for detection and segmentation of disease– In Screening most of the patients are normal (80-90% for DR & BC)
• Multiple tools result in cascading effect of detected FPs• Doctors spend a lot of time in rejecting normal patients
– Other challenges in disease centric approach• Illumination and Contrast• Tissue Pigmentation
• A disease centric CAD system has to robustly learn all possible manifestations of a disease which is challenging
• Patients with diseases outside the purview of screening are ignored – referral could be useful for a patient suffering non DR disease detected in DR screening
Disease ScreeningDisease Screening
PART III – Proposed Methodology
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
• Non conformance to expected behaviour (normal) in the data is considered as abnormality
• Features of normal medical images can be used to model expected normal behaviour
• Abnormality detection is relevant in disease screening where detecting the presence of abnormality is of initial interest:– Retinal image screening for detecting Diabetic
Retinopathy– Mammographic screening for detecting
malignancy of lesions Normal CFI Abnormal CFI with lesions
Proposed MethodologyProposed Methodology
Detecting Abnormality instead of Disease
X
Y
Normal
Abnormal
Feature Space
Abnormal
Two Stage Methodology for CAD
Proposed MethodologyProposed Methodology
• Stage 1- Detection of abnormality – Derive motion pattern for detection of
lesions– Extract relevant features to represent
normal sub-space– Detect outliers as abnormal
• Stage2-Assessment of abnormality– Derive relevant features based on
domain knowledge from abnormal cases
– Determine the severity of disease
Two Stage Methodology for CAD
Proposed MethodologyProposed Methodology
• Stage 1- Detection of abnormality – Only normal cases are required for
disease detection– Variations observed in the normal
cases are captured by the normal feature sub-space
– Single point of control on the permitted figure of false alarms
• Stage2-Assessment of abnormality– Fewer normal cases to be examined
by experts
• Motivation - Effect of motion on human visual system and detectors in camera– Spatial/temporal averaging of intensities in retina– Smearing of intensities corresponding to moving
object is observed in images acquired with camera
• Inducing motion in images– Lesions can be observed as a set of localized pixels
with contrast against background– A smear of pixel along the direction of motion can
be observed in motion pattern– Spread and extent of lesions in motion pattern
depends on the sampling rate at each location and duration of motion
– Contrast of the spatially enhanced lesions in motion pattern relies on the coalescing function
• Motion pattern on Background– Uniformity in motion pattern for textured
background can be observed
Orig
inal
Imag
e (U
nifo
rm B
ackg
roun
d)Ro
tatio
nal M
otion
Patt
ern
Motion Pattern – Detecting Localized Lesions
Proposed MethodologyProposed MethodologyO
riginal Image
(Textured Background)Rotational M
otion Pattern
PART IV – Detection and Assessment of Macular Edema
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Macular Edema Detection and Assessment• Diabetic Macular Edema (DME) is a sight threatening condition that
occurs due to diabetic retinopathy• DME requires immediate referral to Ophthalmologists• Presence of Hard Exudates is used as an indicator of DME during retinal
disease screening
Severe and moderate cases of DMEColor Retinal Image
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Existing Approaches in DME Detection• Several local and global schemes have been proposed for DME
detection
• Local Schemes– local schemes try to successfully segment or localize the exudate clusters– Techniques including adaptive intensity thresholding, background suppression
(median filtering, morphology), color and edge detection have been proposed – several normal pixels are also detected as candidates in normal images increasing the
number of false alarms in the system
• Global Schemes– global schemes try to ensure that at least the brightest pixels corresponding to HE in
the image are detected– Techniques based on intensity thresholding, edge strength, and visual words using
features on SIFT keypoints have been used to classify images
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Proposed Workflow
Steps• Landmark Detection and Region of Interest Extraction• Generation of Motion Patterns• Feature Selection• Abnormality Detection• Abnormality Assessment
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Detection of Landmarks in CFI-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Singh, J. and Joshi, G. D. and Sivaswamy, J. Appearance-based object detection in colour retinal images. In ICIP, pages 1432–1435, 2008.G. D. Joshi and J. Sivaswamy and K Karan and S. R. Krishnadas. Optic disk and cup boundary detection using regional information. ISBI, pp. 948–951, 2010.
Motion Pattern – Rotational Motion
Effect of sampling rate on motion pattern (decreasing rotation steps)-
Coalescing Function• Mean - Arithmetic mean of all samples were taken
• Extrema – Maximum or Minimum of all samples are taken at each pixel location
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Selection of Motion Pattern
“effect of abnormality (lesion) on retinal background can be observed as change in local information with respect to the motion pattern of normal retina”
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
normal abnormal• A normal retinal image was created by averaging the green channel of 400 retinal images• The abnormal retina is modeled by adding a bright lesion to emulate HE
- motion pattern - Gradient magnitude of motion pattern - Shannon’s entropy
Selection of Parameters – Class Discriminability-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Size of normal retina – 150*150Neighborhood size – 7*7
Motion Pattern for Edema Detection• A circular ROI is determined around macula and the Optic disc is masked to
avoid false positives• Rotational motion is induced in the green channel image• Maxima is used as the coalescing function• Features derived on motion pattern are used for learning the normal sub-
space and detecting abnormality
Sample ROI and Motion Pattern (S- Subtle Hard Exudates)
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
More Motion PatternsSample ROIs and Motion Pattern (S- Subtle Hard Exudates)
Nor
mal
RO
IAb
norm
al R
OI
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Feature Extraction-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
• To effectively describe motion pattern, we use a descriptor derived from the Radon space
• The desired feature vector is obtained by concatenating 6 projections (0-180 degrees) • Each projection has 6 bins resulting in a feature vector of length 36
Integral of motion pattern along a line
Abnormality DetectionPCA Data Description• The eigenvectors corresponding to the covariance matrix of the training set is used to describe the normal subspace• Feature vector for a new case is projected to this subspace (first 6 eigen vectors)
Residual e is defined as,
• Classification between normal and abnormal cases is then performed using an empirically determined threshold on e
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
FNTP
TPySensitivit
FPTN
TNySpecificit
Detection Performance (ROC Curves)• DMED - 122 images
o Normal - 68o Abnormal – 54o Normal images used for training - 18
• MESSIDOR – 400 imageso Normal - 274o Abnormal – 126o Immediate referral - 85o Normal images used for training – 74
• Diaretdb0 & db1 – 122 imageso Normal – 25o Abnormal - 97
• Combined Dataset – 644 imageso Normal – 367o Abnormal - 277
DM
EDM
ESSI
DO
R
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Receiver Operating Characteristic curve
Comparison against Disease Centric MethodsDMED
Normal - 68Abnormal – 54Normal images used for training - 18
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
MESSIDORNormal - 274Abnormal – 126Normal images used for training – 74
[23] L. Giancardo, F. Meriaudeau, T. P. Karnowski, Y. Li, K. W. Tobin Jr, and E. Chaum. Automatic retina exudates segmentation without a manually labelled training set. IEEE ISBI, pages 1396 – 1400, April 2011.
[2] C. Agurto, V. Murray, E. Barriga, S. Murillo, M. Pattichis, H. Davis, S. Russell, M. Abramoff, and P. Soliz. Multiscale am-fm methods for diabetic retinopathy lesion detection. IEEE TMI, 29(2):502 –512, feb. 2010.
Assessment of Severity
• Macula is devoid of significant vasculature • It is characterized by rough rotationally symmetry
-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
- Abnormal image
- Symmetry measure on abnormal macula
and are the minimum and maximum symmetry values for normal cases
Assessment of Severity-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Dataset: MESSIDOR
The threshold is expressed as a percentage (p) of the symmetry measure S of normal ROIs used in the abnormality detection task
Detection of Multiple Abnormalities-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
Normal Cases - 362 Abnormal Cases - 302
Dataset: DMED,MESSIDOR and Diaretdb0
Abnormalities: Hemorrhage, Hard Exudates, Drusen
PART V – Classification of Lesions in Mammograms
Computer Aided Diagnosis
Computer Aided Diagnosis Disease ScreeningDisease Screening Proposed MethodologyProposed Methodology -Showcase 1-
Retinopathy-Showcase 1- Retinopathy
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Assessment of Mammographic Lesions• Breast cancer is responsible for about 30 percent of all new cancer cases
with a high mortality rate in women• Screening for its early detection with mammograms has been explored for
more than 3 decades now with moderate success• Correct classification of anomalous areas in the mammograms through
visual examination is challenging even for experts
Sample Benign and Malignant lesions in Mammograms
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Existing Approaches in Mammogram Analysis• 1- Lesions are first detected from mammograms• 2- Malignancy of detected lesions are identified using several texture
and shape features
• Typical features used– size– shape– density– Smoothness of borders– Brightness and contrast– local intensity distribution
• The feature space is very large and complex due to the wide diversity of the normal tissues and the variety of the abnormalities
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Classification of Mammographic Lesions• Given a lesion, its malignancy is of question• Features derived over motion pattern is used for learning the behavior of
benign class• Any deviation in lesion property is identified as a sign of malignancy
Benign lesions
Malignant lesions
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Motion Pattern – Class Discriminability• Three sample benign and malignant lesions were selected • Motion pattern was applied using rotation and translation to analyze class discriminability between benign and malignant class
• Maximum and Mean are the coalescing functions used
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
Classification Performance (ROC Curve)
• An evaluation of the proposed scheme for learning normal subspace was conducted using KNN classifier
• The value of K was considered as 3 for computing the sensitivity and specificity values in the classification tasks
• An ROC curve is drawn by varying the normalized Euclidean distance from [0-1]
Mini-MIAS
Benign - 68Malignant – 51Benign lesions for training - 20
-Showcase 2- Breast Cancer-Showcase 2- Breast Cancer
• We identified and listed the challenges in image based disease screening for diabetic retinopathy and breast cancer
• We proposed and evaluated a method for abnormality detection and assessment– a hierarchical approach to the problem of abnormality detection
• Evaluation of the proposed hierarchical approach has been performed – on several publicly image datasets of CFI and mammograms– improvement in the disease detection performance over methods in literature
Conclusion
Acknowledgement
• This work is dedicated to my Parents and Teachers
• Extremely grateful to Prof. Jayanthi Sivaswamy for giving me the opportunity to pursue MS by research
• Thankful to all lab mates in CVIT for their support• Guidance of Gopal and Mayank was extremely valuable • Debates and discussion with Sandeep, Kartheek and Saurabh were always
insightful
Publications1. Patents
(a) Jayanthi Sivaswamy, N V Kartheek Medathati, K Sai Deepak, A System for generating Generalized Moment Patterns, Submitted to Indian Patent Office, 2010 (Application Number 3939-CHE-2010)
2. Papers
Conference(a) K Sai Deepak, Gopal Datt Joshi, Jayanthi Sivaswamy, Content-Based Retrieval of Retinal Images for Maculopathy, ACM International Health Informatics Symposium, November, 2010
Journal(a) K Sai Deepak, N V Kartheek Medathati and Jayanthi Sivaswamy, Detection and Discrimination of disease related abnormalities, Elsevier Pattern Recognition 2011 (In Press)(b) K Sai Deepak, Jayanthi Sivaswamy, Automatic Assessment of Macular Edema from Color Retinal Images, IEEE Transactions on Medical Imaging 2011
Imaging Modalities
Computer Aided Diagnosis
Computer Aided Diagnosis
Optical Imaging - Ophthalmology X-ray Imaging - Mammography
• High resolution optical camera• Pupil may be dilated before imaging• Pixel resolutions typically range from 0.5K to ~2K*2K• Radiometric resolution is typically 8 bits per channel
• Low energy X-ray scanner• Displays change of density among tissues• Pixel resolutions can range from 1K2 to 3K2
• Radiometric resolution 8-12 bits
CAD in Disease Screening – Diabetic Retinopathy
Disease ScreeningDisease Screening
Hemorrhage Detection
Exudate Detection
Neovascularization Detection
MicroaneurysmsDetection
FP1
FP2
FP3
FP4
Maximum False alarms in disease centric approach – FP1 + FP2 + FP3 + FP4
Illumination and Contrast
Disease ScreeningDisease Screening
• Presence of one or more of additive bias, multiplicative bias and difference in brightness
• These variations often increases the complexity of modeling the normal background especially when there can be several other structures present in the normal image
Tissue Variation (Pigmentation & Density)
Disease ScreeningDisease Screening
• Tissue characteristics for the same structure can vary across race and often across patients, within a race.
• This variation manifests as differences in intensity, hue and/or pigmentation• These variations can be significant enough for an automated disease detection
technique to classify an image as abnormal
CAD with Images - Visualization
Computer Aided Diagnosis
Computer Aided Diagnosis
MAP of Sagittal view Bones appear bright in X-ray
52 year old Patient with Back Pain
WindowingTissues of varying densities can be examined
CAD with Images - Detection
Computer Aided Diagnosis
Computer Aided Diagnosis
Normal Retina Abnormal Retina
CAD with Images – Segmentation
Computer Aided Diagnosis
Computer Aided Diagnosis
Original Image Vessels Segmented
Feature Extraction-Showcase 1- Retinopathy
-Showcase 1- Retinopathy
• To effectively describe motion pattern, we use a descriptor derived from the Radon space
- is the integral of motion pattern along a line oriented at and distance from the origin
The desired feature vector is obtained by concatenating projections from each bin at different orientations