An investigation on combination methods

download An investigation on combination methods

of 42

  • date post

    19-Dec-2014
  • Category

    Technology

  • view

    259
  • download

    2

Embed Size (px)

description

 

Transcript of An investigation on combination methods

  • 1. An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval ALI HOSSEINZADEH VAHID ASST.PROF.DR.ADIL ALPKOAK AUGUST, 2012 ZMR

2. Introduction 2 Medical images are playing an important role todetect anatomical and functional information of the body part for diagnosis, medical research and education : physicians or radiologists examine them in conventional ways based on their individual experiences and knowledge provide diagnostic support to physicians or radiologists by displaying relevant past cases. as a training tool for medical students and residents in education, follow-up studies, and for research purposes.An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 3. Background(Image Retrieval Systems) 3 Image retrieval is a poor stepchild to other formsof information retrieval (IR). Image retrieval has been one of the most interesting and vivid research areas in the field of computer vision over the last decades. An image retrieval system is a computer system for browsing, searching and retrieving similar images (may not be exact) from a large database of digital images with the help of some key attributes associated with the images or features inherently contained in the images. An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 4. Background(TBIR) 4 In Text Based Image Retrieval (TBIR)system,images are indexed by text, known as the metadata of the image, such as the patients ID number, the date it was produced, the type of the image and a manually annotated description on the content of the image itself such as Google Images and Flickr. image retrieval based only on text information is notsufficient since : The amount of labor required to manually annotate every single image, The difference in human perception when describing the images, which might lead to inaccuracies during the retrieval process.An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 5. Background(CBIR) 5 The main goal in Content Base Image Retrieval system is searchingand finding similar images based on their content. To accomplish this, the content should first be described in an efficientway, e.g. the so-called indexing or feature extraction and binary signatures are formed and stored as the data When the query image is given to the system, the system will extract imagefeatures for this query. It will compare these features with that of other images in a database. Relevant results will be displayed to the user. There are many factors to consider in the design of a CBIR: Choice of right features: how to mathematically describe an image ? Similarity measurement criteria: how to assess the similarity between a pair of images? Indexing mechanism and Query formulation techniqueAn Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 6. Background (CBIR) 6 Major problems of CBIR are : Semantic gap: The lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data have for a user in a given situation. User seeks semantic similarity, but the database can only provide similarity by data processing.Huge amount of objects to search among. Incomplete query specification. Incomplete image description.An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 7. Image Content Descriptors 7 image content may include : Visual content General : include color, texture, shape, spatial relationship, etc. Domain specific: is application dependent and may involve domain knowledge Semantic content is obtained by textual annotation by complex inference procedures based on visual contentAn Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 8. Color 8 One of the most widely used visual features Relatively robust to changes in the background colors Independent of image size and orientation Considerable design and experimental work in MPEG-7 to arrive at efficient color descriptors for similarity matching. No single generic color descriptor exists that can be used for all foreseen applications. Such as SCD, CLD, CSDAn Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 9. Texture 9 Another fundamental visual feature This contains structure ness, regularity, directionality and roughness of images Such as HTD, EHDAn Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 10. Compact composite descriptors 10 Color and edge directivity descriptor (CEDD) The six-bin histogram of the fuzzy system that uses the five digital filters proposed by the MPEG-7 EHD. The 24-bin color histogram produced by the 24-bin fuzzylinking system. Overall, the final histogram has 144 regions. Fuzzy color and texture histogram (FCTH) The eight-bin histogram of the fuzzy system that uses the high frequency bands of the Haar wavelet transform The 24-bin color histogram produced by the 24-bin fuzzylinking system. Overall, the final histogram includes192 regions. An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 11. Compact composite descriptors 11 Brightness and Texture Directionality Histogram BTDH is very similar to FCTH feature. The main difference is using brightness instead of color histogram. uses brightness and texture characteristics as well as the spatial distribution of these characteristics in one compact 1D vector. The texture information comes from the Directionality histogram. Fractal Scanning method through the Hilbert Curve or the ZGrid method is used to capture the spatial distribution of brightness and texture information An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 12. Similarity Measures 12 Geometric Measures treat objects as vectors. Information Theoretic Measures are derived fromthe Shannons entropy theory and treat objects as probabilistic distributions Statistic Measures compare two objects in adistributed manner, and basically assume that the vector elements are samples. An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 13. Performance evaluation 13An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 14. Performance evaluation 14An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 15. Need to fuse (CBIR) 15 Some research efforts have been reported to enhanceCBIR performance by taking the multi-modality fusion approaches: Since each feature extracted from images just characterizes certain aspect of image content.A special feature is not equally important for different image queries since a special feature has different importance in reflecting the content of different images.An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 16. Fusion 16 Information fusion is the study of efficient methodsfor automatically or semi-automatically transforming information from different sources and different points in time into a representation that provides effective support for human or automated decision making. The major challenge is to find adjusted techniques for associating multiple sources of information for either decisionmaking or information retrieval. traditional work on multimodal integration has largely been heuristic-based. Still today, the understanding of how fusion works and by what it is influenced is limited.An Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 17. Significant techniques in the multimodal fusion process 17 Feature level fusion: An information process thatintegrates, associates, correlates and combines unimodal features, data and information from single or multiple sensors or sources to achieve refined estimates of parameters, characteristics, events and behaviors The information fusion at data or sensor level can achieve the best performance improvements (Koval, 2007) . It can utilize the correlation between multiple features from different modalities at an early stage which helps in better task accomplishment. Also, it requires only one learning phase on the combined feature vector it is hard to represent the time synchronization between the multimodal features. The features to be fused should be represented in the same format before fusion. The increase in the number of modalities makes it difficult to learn the crosscorrelation among the heterogeneous featuresAn Investigation on Combination Methods for Multimodal Content-based Medical Image Retrieval10/8/2012 18. Significant techniques in the multimodal fusion process 18 Score, rank and decision level fusion, also called high-level, late information fusion, arose in the neural network literature. Here, each modality/ sensor/ source/ feature is first processed individually. The results, so called experts, can be scores in classification or ranks for retrieval. The expert's values are then combined for determining the final decision. This type of information fusion is faster and easier to implement than early fusion. The decision level fusion strategy offers scalability (i.e. graceful upgrading or degrading) in terms of the modalities used in the fusion process. The disadvantage of the late fusion approach lies in its failure to utilize the feature level correlation among modalities As different classifiers are used to obtain the local decisions, the learning process for them becomes tedious and time-consuming.An Investigation on Combination Methods for Multimodal Content-based Medical I